diff --git a/.github/workflows/shellcheck.yml b/.github/workflows/shellcheck.yml new file mode 100644 index 0000000000..68d12b805f --- /dev/null +++ b/.github/workflows/shellcheck.yml @@ -0,0 +1,19 @@ +name: Shellcheck + +on: + push: + branches-ignore: + - 'release/*' + pull_request: + +jobs: + shellcheck: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - name: Install shellcheck + run: sudo apt update && sudo apt install --assume-yes shellcheck + + - name: Run shellcheck + run: find . -name \*.sh | xargs shellcheck --severity=warning diff --git a/.readthedocs.yml b/.readthedocs.yml index 3dcf0e5cf6..2a3c920b45 100644 --- a/.readthedocs.yml +++ b/.readthedocs.yml @@ -6,9 +6,10 @@ sphinx: configuration: docs/conf.py build: - image: latest + os: "ubuntu-22.04" + tools: + python: "3.8" python: - version: 3.8 install: - requirements: docs-requirements.txt diff --git a/CHANGELOG.md b/CHANGELOG.md index 93a8e9b917..193779659c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,10 +6,9 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ## Unreleased + ### Added -- `opentelemetry-sdk-extension-aws` Register AWS resource detectors under the - `opentelemetry_resource_detector` entry point - ([#2382](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/2382)) + - `opentelemetry-instrumentation-pika` Instrumentation for `channel.consume()` (supported only for global, non channel specific instrumentation) ([#2397](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/2397))) @@ -19,9 +18,32 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Rename `type` attribute to `asgi.event.type` in `opentelemetry-instrumentation-asgi` ([#2300](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/2300)) +- Rename AwsLambdaInstrumentor span attributes `faas.id` to `cloud.resource_id`, `faas.execution` to `faas.invocation_id` + ([#2372](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/2372)) +- Drop support for instrumenting elasticsearch client < 6` + ([#2422](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/2422)) ### Added +- `opentelemetry-sdk-extension-aws` Register AWS resource detectors under the + `opentelemetry_resource_detector` entry point + ([#2382](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/2382)) +- `opentelemetry-instrumentation-wsgi` Implement new semantic convention opt-in with stable http semantic conventions + ([#2425](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/2425)) +- `opentelemetry-instrumentation-threading` Initial release for threading + ([#2253](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/2253)) + +### Fixed + +- `opentelemetry-instrumentation-grpc` AioClientInterceptor should propagate with a Metadata object + ([#2363](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/2363)) + +### Added + +- `opentelemetry-sdk-extension-aws` Register AWS resource detectors under the `opentelemetry_resource_detector` entry point + ([#2382](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/2382)) +- `opentelemetry-instrumentation-wsgi` Implement new semantic convention opt-in with stable http semantic conventions + ([#2425](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/2425)) - `opentelemetry-instrumentation-threading` Initial release for threading ([#2253](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/2253)) diff --git a/README.md b/README.md index 6968de717e..133acf81f6 100644 --- a/README.md +++ b/README.md @@ -96,11 +96,15 @@ To release a package as `1.0` stable, the package: - e.g. If an instrumentation package uses flags, a token as context, or parameters that are not typical of the `BaseInstrumentor` class, these are documented - After the release of `1.0`, a CODEOWNER may no longer feel like they have the bandwidth to meet the responsibilities of maintaining the package. That's not a problem at all, life happens! However, if that is the case, we ask that the CODEOWNER please raise an issue indicating that they would like to be removed as a CODEOWNER so that they don't get pinged on future PRs. Ultimately, we hope to use that issue to find a new CODEOWNER. +## Semantic Convention status of instrumentations + +In our efforts to maintain optimal user experience and prevent breaking changes for transitioning into stable semantic conventions, OpenTelemetry Python is adopting the [semantic convention migration plan](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/http/migration-guide.md) for several instrumentations. Currently this plan is only being adopted for HTTP-related instrumentations, but will eventually cover all types. Please refer to the `semconv status` column of the [instrumentation README](instrumentation/README.md) of the current status of instrumentations' semantic conventions. The possible values are `experimental`, `stable` and `migration` referring to [status](https://github.com/open-telemetry/opentelemetry-specification/blob/v1.31.0/specification/document-status.md#lifecycle-status) of that particular semantic convention. `Migration` refers to an instrumentation that currently supports the migration plan. + ## Contributing See [CONTRIBUTING.md](CONTRIBUTING.md) -We meet weekly on Thursday, and the time of the meeting alternates between 9AM PT and 4PM PT. The meeting is subject to change depending on contributors' availability. Check the [OpenTelemetry community calendar](https://calendar.google.com/calendar/embed?src=google.com_b79e3e90j7bbsa2n2p5an5lf60%40group.calendar.google.com) for specific dates and for the Zoom link. +We meet weekly on Thursday at 9AM PT. The meeting is subject to change depending on contributors' availability. Check the [OpenTelemetry community calendar](https://calendar.google.com/calendar/embed?src=c_2bf73e3b6b530da4babd444e72b76a6ad893a5c3f43cf40467abc7a9a897f977%40group.calendar.google.com) for specific dates and for the Zoom link. Meeting notes are available as a public [Google doc](https://docs.google.com/document/d/1CIMGoIOZ-c3-igzbd6_Pnxx1SjAkjwqoYSUWxPY8XIs/edit). For edit access, get in touch on [GitHub Discussions](https://github.com/open-telemetry/opentelemetry-python/discussions). diff --git a/docs-requirements.txt b/docs-requirements.txt index aff449fcf8..72f4472902 100644 --- a/docs-requirements.txt +++ b/docs-requirements.txt @@ -28,7 +28,7 @@ boto3~=1.0 cassandra-driver~=3.25 celery>=4.0 confluent-kafka>= 1.8.2,<= 2.3.0 -elasticsearch>=2.0,<9.0 +elasticsearch>=6.0,<9.0 flask~=2.0 falcon~=2.0 grpcio~=1.27 diff --git a/exporter/opentelemetry-exporter-prometheus-remote-write/proto/generate-proto-py.sh b/exporter/opentelemetry-exporter-prometheus-remote-write/proto/generate-proto-py.sh index 5f6129df55..a07181add3 100755 --- a/exporter/opentelemetry-exporter-prometheus-remote-write/proto/generate-proto-py.sh +++ b/exporter/opentelemetry-exporter-prometheus-remote-write/proto/generate-proto-py.sh @@ -6,7 +6,6 @@ PROTO_VERSION=v1.3.2 # SRC_DIR is from protoc perspective. ie its the destination for our checkouts/clones SRC_DIR=opentelemetry/exporter/prometheus_remote_write/gen/ -DST_DIR=../src/opentelemetry/exporter/prometheus_remote_write/gen/ #TODO: # Check that black & protoc are installed properly diff --git a/instrumentation/README.md b/instrumentation/README.md index 62caabd81a..284037d707 100644 --- a/instrumentation/README.md +++ b/instrumentation/README.md @@ -1,51 +1,51 @@ -| Instrumentation | Supported Packages | Metrics support | -| --------------- | ------------------ | --------------- | -| [opentelemetry-instrumentation-aio-pika](./opentelemetry-instrumentation-aio-pika) | aio_pika >= 7.2.0, < 10.0.0 | No -| [opentelemetry-instrumentation-aiohttp-client](./opentelemetry-instrumentation-aiohttp-client) | aiohttp ~= 3.0 | No -| [opentelemetry-instrumentation-aiohttp-server](./opentelemetry-instrumentation-aiohttp-server) | aiohttp ~= 3.0 | No -| [opentelemetry-instrumentation-aiopg](./opentelemetry-instrumentation-aiopg) | aiopg >= 0.13.0, < 2.0.0 | No -| [opentelemetry-instrumentation-asgi](./opentelemetry-instrumentation-asgi) | asgiref ~= 3.0 | No -| [opentelemetry-instrumentation-asyncio](./opentelemetry-instrumentation-asyncio) | asyncio | No -| [opentelemetry-instrumentation-asyncpg](./opentelemetry-instrumentation-asyncpg) | asyncpg >= 0.12.0 | No -| [opentelemetry-instrumentation-aws-lambda](./opentelemetry-instrumentation-aws-lambda) | aws_lambda | No -| [opentelemetry-instrumentation-boto](./opentelemetry-instrumentation-boto) | boto~=2.0 | No -| [opentelemetry-instrumentation-boto3sqs](./opentelemetry-instrumentation-boto3sqs) | boto3 ~= 1.0 | No -| [opentelemetry-instrumentation-botocore](./opentelemetry-instrumentation-botocore) | botocore ~= 1.0 | No -| [opentelemetry-instrumentation-cassandra](./opentelemetry-instrumentation-cassandra) | cassandra-driver ~= 3.25,scylla-driver ~= 3.25 | No -| [opentelemetry-instrumentation-celery](./opentelemetry-instrumentation-celery) | celery >= 4.0, < 6.0 | No -| [opentelemetry-instrumentation-confluent-kafka](./opentelemetry-instrumentation-confluent-kafka) | confluent-kafka >= 1.8.2, <= 2.3.0 | No -| [opentelemetry-instrumentation-dbapi](./opentelemetry-instrumentation-dbapi) | dbapi | No -| [opentelemetry-instrumentation-django](./opentelemetry-instrumentation-django) | django >= 1.10 | Yes -| [opentelemetry-instrumentation-elasticsearch](./opentelemetry-instrumentation-elasticsearch) | elasticsearch >= 2.0 | No -| [opentelemetry-instrumentation-falcon](./opentelemetry-instrumentation-falcon) | falcon >= 1.4.1, < 4.0.0 | Yes -| [opentelemetry-instrumentation-fastapi](./opentelemetry-instrumentation-fastapi) | fastapi ~= 0.58 | Yes -| [opentelemetry-instrumentation-flask](./opentelemetry-instrumentation-flask) | flask >= 1.0 | Yes -| [opentelemetry-instrumentation-grpc](./opentelemetry-instrumentation-grpc) | grpcio ~= 1.27 | No -| [opentelemetry-instrumentation-httpx](./opentelemetry-instrumentation-httpx) | httpx >= 0.18.0 | No -| [opentelemetry-instrumentation-jinja2](./opentelemetry-instrumentation-jinja2) | jinja2 >= 2.7, < 4.0 | No -| [opentelemetry-instrumentation-kafka-python](./opentelemetry-instrumentation-kafka-python) | kafka-python >= 2.0 | No -| [opentelemetry-instrumentation-logging](./opentelemetry-instrumentation-logging) | logging | No -| [opentelemetry-instrumentation-mysql](./opentelemetry-instrumentation-mysql) | mysql-connector-python ~= 8.0 | No -| [opentelemetry-instrumentation-mysqlclient](./opentelemetry-instrumentation-mysqlclient) | mysqlclient < 3 | No -| [opentelemetry-instrumentation-pika](./opentelemetry-instrumentation-pika) | pika >= 0.12.0 | No -| [opentelemetry-instrumentation-psycopg](./opentelemetry-instrumentation-psycopg) | psycopg >= 3.1.0 | No -| [opentelemetry-instrumentation-psycopg2](./opentelemetry-instrumentation-psycopg2) | psycopg2 >= 2.7.3.1 | No -| [opentelemetry-instrumentation-pymemcache](./opentelemetry-instrumentation-pymemcache) | pymemcache >= 1.3.5, < 5 | No -| [opentelemetry-instrumentation-pymongo](./opentelemetry-instrumentation-pymongo) | pymongo >= 3.1, < 5.0 | No -| [opentelemetry-instrumentation-pymysql](./opentelemetry-instrumentation-pymysql) | PyMySQL < 2 | No -| [opentelemetry-instrumentation-pyramid](./opentelemetry-instrumentation-pyramid) | pyramid >= 1.7 | Yes -| [opentelemetry-instrumentation-redis](./opentelemetry-instrumentation-redis) | redis >= 2.6 | No -| [opentelemetry-instrumentation-remoulade](./opentelemetry-instrumentation-remoulade) | remoulade >= 0.50 | No -| [opentelemetry-instrumentation-requests](./opentelemetry-instrumentation-requests) | requests ~= 2.0 | Yes -| [opentelemetry-instrumentation-sklearn](./opentelemetry-instrumentation-sklearn) | scikit-learn ~= 0.24.0 | No -| [opentelemetry-instrumentation-sqlalchemy](./opentelemetry-instrumentation-sqlalchemy) | sqlalchemy | Yes -| [opentelemetry-instrumentation-sqlite3](./opentelemetry-instrumentation-sqlite3) | sqlite3 | No -| [opentelemetry-instrumentation-starlette](./opentelemetry-instrumentation-starlette) | starlette ~= 0.13.0 | Yes -| [opentelemetry-instrumentation-system-metrics](./opentelemetry-instrumentation-system-metrics) | psutil >= 5 | No -| [opentelemetry-instrumentation-threading](./opentelemetry-instrumentation-threading) | threading | No -| [opentelemetry-instrumentation-tornado](./opentelemetry-instrumentation-tornado) | tornado >= 5.1.1 | Yes -| [opentelemetry-instrumentation-tortoiseorm](./opentelemetry-instrumentation-tortoiseorm) | tortoise-orm >= 0.17.0 | No -| [opentelemetry-instrumentation-urllib](./opentelemetry-instrumentation-urllib) | urllib | Yes -| [opentelemetry-instrumentation-urllib3](./opentelemetry-instrumentation-urllib3) | urllib3 >= 1.0.0, < 3.0.0 | Yes -| [opentelemetry-instrumentation-wsgi](./opentelemetry-instrumentation-wsgi) | wsgi | Yes \ No newline at end of file +| Instrumentation | Supported Packages | Metrics support | Semconv status | +| --------------- | ------------------ | --------------- | -------------- | +| [opentelemetry-instrumentation-aio-pika](./opentelemetry-instrumentation-aio-pika) | aio_pika >= 7.2.0, < 10.0.0 | No | experimental +| [opentelemetry-instrumentation-aiohttp-client](./opentelemetry-instrumentation-aiohttp-client) | aiohttp ~= 3.0 | No | experimental +| [opentelemetry-instrumentation-aiohttp-server](./opentelemetry-instrumentation-aiohttp-server) | aiohttp ~= 3.0 | No | experimental +| [opentelemetry-instrumentation-aiopg](./opentelemetry-instrumentation-aiopg) | aiopg >= 0.13.0, < 2.0.0 | No | experimental +| [opentelemetry-instrumentation-asgi](./opentelemetry-instrumentation-asgi) | asgiref ~= 3.0 | No | experimental +| [opentelemetry-instrumentation-asyncio](./opentelemetry-instrumentation-asyncio) | asyncio | No | experimental +| [opentelemetry-instrumentation-asyncpg](./opentelemetry-instrumentation-asyncpg) | asyncpg >= 0.12.0 | No | experimental +| [opentelemetry-instrumentation-aws-lambda](./opentelemetry-instrumentation-aws-lambda) | aws_lambda | No | experimental +| [opentelemetry-instrumentation-boto](./opentelemetry-instrumentation-boto) | boto~=2.0 | No | experimental +| [opentelemetry-instrumentation-boto3sqs](./opentelemetry-instrumentation-boto3sqs) | boto3 ~= 1.0 | No | experimental +| [opentelemetry-instrumentation-botocore](./opentelemetry-instrumentation-botocore) | botocore ~= 1.0 | No | experimental +| [opentelemetry-instrumentation-cassandra](./opentelemetry-instrumentation-cassandra) | cassandra-driver ~= 3.25,scylla-driver ~= 3.25 | No | experimental +| [opentelemetry-instrumentation-celery](./opentelemetry-instrumentation-celery) | celery >= 4.0, < 6.0 | No | experimental +| [opentelemetry-instrumentation-confluent-kafka](./opentelemetry-instrumentation-confluent-kafka) | confluent-kafka >= 1.8.2, <= 2.3.0 | No | experimental +| [opentelemetry-instrumentation-dbapi](./opentelemetry-instrumentation-dbapi) | dbapi | No | experimental +| [opentelemetry-instrumentation-django](./opentelemetry-instrumentation-django) | django >= 1.10 | Yes | experimental +| [opentelemetry-instrumentation-elasticsearch](./opentelemetry-instrumentation-elasticsearch) | elasticsearch >= 2.0 | No | experimental +| [opentelemetry-instrumentation-falcon](./opentelemetry-instrumentation-falcon) | falcon >= 1.4.1, < 4.0.0 | Yes | experimental +| [opentelemetry-instrumentation-fastapi](./opentelemetry-instrumentation-fastapi) | fastapi ~= 0.58 | Yes | experimental +| [opentelemetry-instrumentation-flask](./opentelemetry-instrumentation-flask) | flask >= 1.0 | Yes | experimental +| [opentelemetry-instrumentation-grpc](./opentelemetry-instrumentation-grpc) | grpcio ~= 1.27 | No | experimental +| [opentelemetry-instrumentation-httpx](./opentelemetry-instrumentation-httpx) | httpx >= 0.18.0 | No | experimental +| [opentelemetry-instrumentation-jinja2](./opentelemetry-instrumentation-jinja2) | jinja2 >= 2.7, < 4.0 | No | experimental +| [opentelemetry-instrumentation-kafka-python](./opentelemetry-instrumentation-kafka-python) | kafka-python >= 2.0 | No | experimental +| [opentelemetry-instrumentation-logging](./opentelemetry-instrumentation-logging) | logging | No | experimental +| [opentelemetry-instrumentation-mysql](./opentelemetry-instrumentation-mysql) | mysql-connector-python ~= 8.0 | No | experimental +| [opentelemetry-instrumentation-mysqlclient](./opentelemetry-instrumentation-mysqlclient) | mysqlclient < 3 | No | experimental +| [opentelemetry-instrumentation-pika](./opentelemetry-instrumentation-pika) | pika >= 0.12.0 | No | experimental +| [opentelemetry-instrumentation-psycopg](./opentelemetry-instrumentation-psycopg) | psycopg >= 3.1.0 | No | experimental +| [opentelemetry-instrumentation-psycopg2](./opentelemetry-instrumentation-psycopg2) | psycopg2 >= 2.7.3.1 | No | experimental +| [opentelemetry-instrumentation-pymemcache](./opentelemetry-instrumentation-pymemcache) | pymemcache >= 1.3.5, < 5 | No | experimental +| [opentelemetry-instrumentation-pymongo](./opentelemetry-instrumentation-pymongo) | pymongo >= 3.1, < 5.0 | No | experimental +| [opentelemetry-instrumentation-pymysql](./opentelemetry-instrumentation-pymysql) | PyMySQL < 2 | No | experimental +| [opentelemetry-instrumentation-pyramid](./opentelemetry-instrumentation-pyramid) | pyramid >= 1.7 | Yes | experimental +| [opentelemetry-instrumentation-redis](./opentelemetry-instrumentation-redis) | redis >= 2.6 | No | experimental +| [opentelemetry-instrumentation-remoulade](./opentelemetry-instrumentation-remoulade) | remoulade >= 0.50 | No | experimental +| [opentelemetry-instrumentation-requests](./opentelemetry-instrumentation-requests) | requests ~= 2.0 | Yes | migration +| [opentelemetry-instrumentation-sklearn](./opentelemetry-instrumentation-sklearn) | scikit-learn ~= 0.24.0 | No | experimental +| [opentelemetry-instrumentation-sqlalchemy](./opentelemetry-instrumentation-sqlalchemy) | sqlalchemy | Yes | experimental +| [opentelemetry-instrumentation-sqlite3](./opentelemetry-instrumentation-sqlite3) | sqlite3 | No | experimental +| [opentelemetry-instrumentation-starlette](./opentelemetry-instrumentation-starlette) | starlette ~= 0.13.0 | Yes | experimental +| [opentelemetry-instrumentation-system-metrics](./opentelemetry-instrumentation-system-metrics) | psutil >= 5 | No | experimental +| [opentelemetry-instrumentation-threading](./opentelemetry-instrumentation-threading) | threading | No | experimental +| [opentelemetry-instrumentation-tornado](./opentelemetry-instrumentation-tornado) | tornado >= 5.1.1 | Yes | experimental +| [opentelemetry-instrumentation-tortoiseorm](./opentelemetry-instrumentation-tortoiseorm) | tortoise-orm >= 0.17.0 | No | experimental +| [opentelemetry-instrumentation-urllib](./opentelemetry-instrumentation-urllib) | urllib | Yes | experimental +| [opentelemetry-instrumentation-urllib3](./opentelemetry-instrumentation-urllib3) | urllib3 >= 1.0.0, < 3.0.0 | Yes | experimental +| [opentelemetry-instrumentation-wsgi](./opentelemetry-instrumentation-wsgi) | wsgi | Yes | experimental \ No newline at end of file diff --git a/instrumentation/opentelemetry-instrumentation-aws-lambda/src/opentelemetry/instrumentation/aws_lambda/__init__.py b/instrumentation/opentelemetry-instrumentation-aws-lambda/src/opentelemetry/instrumentation/aws_lambda/__init__.py index 3ec22d4c0e..8f29a0ec38 100644 --- a/instrumentation/opentelemetry-instrumentation-aws-lambda/src/opentelemetry/instrumentation/aws_lambda/__init__.py +++ b/instrumentation/opentelemetry-instrumentation-aws-lambda/src/opentelemetry/instrumentation/aws_lambda/__init__.py @@ -340,17 +340,17 @@ def _instrumented_lambda_handler_call( # noqa pylint: disable=too-many-branches if span.is_recording(): lambda_context = args[1] # NOTE: The specs mention an exception here, allowing the - # `ResourceAttributes.FAAS_ID` attribute to be set as a span + # `SpanAttributes.CLOUD_RESOURCE_ID` attribute to be set as a span # attribute instead of a resource attribute. # # See more: - # https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/semantic_conventions/faas.md#example + # https://github.com/open-telemetry/semantic-conventions/blob/main/docs/faas/aws-lambda.md#resource-detector span.set_attribute( - ResourceAttributes.FAAS_ID, + SpanAttributes.CLOUD_RESOURCE_ID, lambda_context.invoked_function_arn, ) span.set_attribute( - SpanAttributes.FAAS_EXECUTION, + SpanAttributes.FAAS_INVOCATION_ID, lambda_context.aws_request_id, ) diff --git a/instrumentation/opentelemetry-instrumentation-aws-lambda/tests/test_aws_lambda_instrumentation_manual.py b/instrumentation/opentelemetry-instrumentation-aws-lambda/tests/test_aws_lambda_instrumentation_manual.py index 9bf6f47d7b..51ebb67ebc 100644 --- a/instrumentation/opentelemetry-instrumentation-aws-lambda/tests/test_aws_lambda_instrumentation_manual.py +++ b/instrumentation/opentelemetry-instrumentation-aws-lambda/tests/test_aws_lambda_instrumentation_manual.py @@ -145,8 +145,8 @@ def test_active_tracing(self): self.assertSpanHasAttributes( span, { - ResourceAttributes.FAAS_ID: MOCK_LAMBDA_CONTEXT.invoked_function_arn, - SpanAttributes.FAAS_EXECUTION: MOCK_LAMBDA_CONTEXT.aws_request_id, + SpanAttributes.CLOUD_RESOURCE_ID: MOCK_LAMBDA_CONTEXT.invoked_function_arn, + SpanAttributes.FAAS_INVOCATION_ID: MOCK_LAMBDA_CONTEXT.aws_request_id, ResourceAttributes.CLOUD_ACCOUNT_ID: MOCK_LAMBDA_CONTEXT.invoked_function_arn.split( ":" )[ diff --git a/instrumentation/opentelemetry-instrumentation-celery/tests/test_tasks.py b/instrumentation/opentelemetry-instrumentation-celery/tests/test_tasks.py index ed4dbb5b1d..3ac6a5a70c 100644 --- a/instrumentation/opentelemetry-instrumentation-celery/tests/test_tasks.py +++ b/instrumentation/opentelemetry-instrumentation-celery/tests/test_tasks.py @@ -125,8 +125,9 @@ def test_task_raises(self): self.assertIn(SpanAttributes.EXCEPTION_STACKTRACE, event.attributes) - self.assertEqual( - event.attributes[SpanAttributes.EXCEPTION_TYPE], "CustomError" + # TODO: use plain assertEqual after 1.25 is released (https://github.com/open-telemetry/opentelemetry-python/pull/3837) + self.assertIn( + "CustomError", event.attributes[SpanAttributes.EXCEPTION_TYPE] ) self.assertEqual( diff --git a/instrumentation/opentelemetry-instrumentation-django/test-requirements-1.txt b/instrumentation/opentelemetry-instrumentation-django/test-requirements-1.txt index 116dc015ec..548db1b8e3 100644 --- a/instrumentation/opentelemetry-instrumentation-django/test-requirements-1.txt +++ b/instrumentation/opentelemetry-instrumentation-django/test-requirements-1.txt @@ -1,7 +1,7 @@ asgiref==3.7.2 attrs==23.2.0 Deprecated==1.2.14 -Django==3.2.24 +Django==3.2.25 importlib-metadata==6.11.0 iniconfig==2.0.0 packaging==23.2 diff --git a/instrumentation/opentelemetry-instrumentation-django/test-requirements-2.txt b/instrumentation/opentelemetry-instrumentation-django/test-requirements-2.txt index ac3b40e16b..0970d2f253 100644 --- a/instrumentation/opentelemetry-instrumentation-django/test-requirements-2.txt +++ b/instrumentation/opentelemetry-instrumentation-django/test-requirements-2.txt @@ -2,7 +2,7 @@ asgiref==3.7.2 attrs==23.2.0 backports.zoneinfo==0.2.1 Deprecated==1.2.14 -Django==4.2.10 +Django==4.2.11 importlib-metadata==6.11.0 iniconfig==2.0.0 packaging==23.2 diff --git a/instrumentation/opentelemetry-instrumentation-django/test-requirements-3.txt b/instrumentation/opentelemetry-instrumentation-django/test-requirements-3.txt index 3bb32c4c6b..d853c05877 100644 --- a/instrumentation/opentelemetry-instrumentation-django/test-requirements-3.txt +++ b/instrumentation/opentelemetry-instrumentation-django/test-requirements-3.txt @@ -1,7 +1,7 @@ asgiref==3.7.2 attrs==23.2.0 Deprecated==1.2.14 -Django==4.2.10 +Django==4.2.11 importlib-metadata==6.11.0 iniconfig==2.0.0 packaging==23.2 diff --git a/instrumentation/opentelemetry-instrumentation-elasticsearch/pyproject.toml b/instrumentation/opentelemetry-instrumentation-elasticsearch/pyproject.toml index 659366181a..f5ba221b7f 100644 --- a/instrumentation/opentelemetry-instrumentation-elasticsearch/pyproject.toml +++ b/instrumentation/opentelemetry-instrumentation-elasticsearch/pyproject.toml @@ -32,7 +32,7 @@ dependencies = [ [project.optional-dependencies] instruments = [ - "elasticsearch >= 2.0", + "elasticsearch >= 6.0", ] [project.entry-points.opentelemetry_instrumentor] diff --git a/instrumentation/opentelemetry-instrumentation-elasticsearch/test-requirements-0.txt b/instrumentation/opentelemetry-instrumentation-elasticsearch/test-requirements-0.txt deleted file mode 100644 index 216d1c0b02..0000000000 --- a/instrumentation/opentelemetry-instrumentation-elasticsearch/test-requirements-0.txt +++ /dev/null @@ -1,22 +0,0 @@ -asgiref==3.7.2 -attrs==23.2.0 -Deprecated==1.2.14 -elasticsearch==2.4.1 -elasticsearch-dsl==2.2.0 -importlib-metadata==6.11.0 -iniconfig==2.0.0 -packaging==23.2 -pluggy==1.4.0 -py==1.11.0 -py-cpuinfo==9.0.0 -pytest==7.1.3 -pytest-benchmark==4.0.0 -python-dateutil==2.8.2 -six==1.16.0 -tomli==2.0.1 -typing_extensions==4.10.0 -urllib3==1.26.18 -wrapt==1.16.0 -zipp==3.17.0 --e opentelemetry-instrumentation --e instrumentation/opentelemetry-instrumentation-elasticsearch diff --git a/instrumentation/opentelemetry-instrumentation-elasticsearch/test-requirements-1.txt b/instrumentation/opentelemetry-instrumentation-elasticsearch/test-requirements-1.txt deleted file mode 100644 index 2c51c87508..0000000000 --- a/instrumentation/opentelemetry-instrumentation-elasticsearch/test-requirements-1.txt +++ /dev/null @@ -1,22 +0,0 @@ -asgiref==3.7.2 -attrs==23.2.0 -Deprecated==1.2.14 -elasticsearch==5.5.3 -elasticsearch-dsl==5.4.0 -importlib-metadata==6.11.0 -iniconfig==2.0.0 -packaging==23.2 -pluggy==1.4.0 -py==1.11.0 -py-cpuinfo==9.0.0 -pytest==7.1.3 -pytest-benchmark==4.0.0 -python-dateutil==2.8.2 -six==1.16.0 -tomli==2.0.1 -typing_extensions==4.10.0 -urllib3==2.2.1 -wrapt==1.16.0 -zipp==3.17.0 --e opentelemetry-instrumentation --e instrumentation/opentelemetry-instrumentation-elasticsearch diff --git a/instrumentation/opentelemetry-instrumentation-elasticsearch/test-requirements-2.txt b/instrumentation/opentelemetry-instrumentation-elasticsearch/test-requirements-6.txt similarity index 100% rename from instrumentation/opentelemetry-instrumentation-elasticsearch/test-requirements-2.txt rename to instrumentation/opentelemetry-instrumentation-elasticsearch/test-requirements-6.txt diff --git a/instrumentation/opentelemetry-instrumentation-elasticsearch/tests/helpers_es2.py b/instrumentation/opentelemetry-instrumentation-elasticsearch/tests/helpers_es2.py deleted file mode 100644 index 008a95d671..0000000000 --- a/instrumentation/opentelemetry-instrumentation-elasticsearch/tests/helpers_es2.py +++ /dev/null @@ -1,33 +0,0 @@ -from elasticsearch_dsl import ( # pylint: disable=no-name-in-module - DocType, - String, -) - - -class Article(DocType): - title = String(analyzer="snowball", fields={"raw": String()}) - body = String(analyzer="snowball") - - class Meta: - index = "test-index" - - -dsl_create_statement = { - "mappings": { - "article": { - "properties": { - "title": { - "analyzer": "snowball", - "fields": {"raw": {"type": "string"}}, - "type": "string", - }, - "body": {"analyzer": "snowball", "type": "string"}, - } - } - }, - "settings": {"analysis": {}}, -} -dsl_index_result = (1, {}, '{"created": true}') -dsl_index_span_name = "Elasticsearch/test-index/article/2" -dsl_index_url = "/test-index/article/2" -dsl_search_method = "GET" diff --git a/instrumentation/opentelemetry-instrumentation-elasticsearch/tests/helpers_es5.py b/instrumentation/opentelemetry-instrumentation-elasticsearch/tests/helpers_es5.py deleted file mode 100644 index cf32d98863..0000000000 --- a/instrumentation/opentelemetry-instrumentation-elasticsearch/tests/helpers_es5.py +++ /dev/null @@ -1,33 +0,0 @@ -from elasticsearch_dsl import ( # pylint: disable=no-name-in-module - DocType, - Keyword, - Text, -) - - -class Article(DocType): - title = Text(analyzer="snowball", fields={"raw": Keyword()}) - body = Text(analyzer="snowball") - - class Meta: - index = "test-index" - - -dsl_create_statement = { - "mappings": { - "article": { - "properties": { - "title": { - "analyzer": "snowball", - "fields": {"raw": {"type": "keyword"}}, - "type": "text", - }, - "body": {"analyzer": "snowball", "type": "text"}, - } - } - }, -} -dsl_index_result = (1, {}, '{"created": true}') -dsl_index_span_name = "Elasticsearch/test-index/article/2" -dsl_index_url = "/test-index/article/2" -dsl_search_method = "GET" diff --git a/instrumentation/opentelemetry-instrumentation-elasticsearch/tests/test_elasticsearch.py b/instrumentation/opentelemetry-instrumentation-elasticsearch/tests/test_elasticsearch.py index 6008108d79..5d1c85f77d 100644 --- a/instrumentation/opentelemetry-instrumentation-elasticsearch/tests/test_elasticsearch.py +++ b/instrumentation/opentelemetry-instrumentation-elasticsearch/tests/test_elasticsearch.py @@ -44,10 +44,6 @@ from . import helpers_es7 as helpers # pylint: disable=no-name-in-module elif major_version == 6: from . import helpers_es6 as helpers # pylint: disable=no-name-in-module -elif major_version == 5: - from . import helpers_es5 as helpers # pylint: disable=no-name-in-module -else: - from . import helpers_es2 as helpers # pylint: disable=no-name-in-module Article = helpers.Article diff --git a/instrumentation/opentelemetry-instrumentation-falcon/tests/test_falcon.py b/instrumentation/opentelemetry-instrumentation-falcon/tests/test_falcon.py index 2245dbfd80..bf7f1d4f49 100644 --- a/instrumentation/opentelemetry-instrumentation-falcon/tests/test_falcon.py +++ b/instrumentation/opentelemetry-instrumentation-falcon/tests/test_falcon.py @@ -16,21 +16,21 @@ from unittest.mock import Mock, patch import pytest -from falcon import __version__ as _falcon_verison +from falcon import __version__ as _falcon_version from falcon import testing from packaging import version as package_version from opentelemetry import trace +from opentelemetry.instrumentation._semconv import ( + _server_active_requests_count_attrs_old, + _server_duration_attrs_old, +) from opentelemetry.instrumentation.falcon import FalconInstrumentor from opentelemetry.instrumentation.propagators import ( TraceResponsePropagator, get_global_response_propagator, set_global_response_propagator, ) -from opentelemetry.instrumentation.wsgi import ( - _active_requests_count_attrs, - _duration_attrs, -) from opentelemetry.sdk.metrics.export import ( HistogramDataPoint, NumberDataPoint, @@ -53,8 +53,8 @@ "http.server.duration", ] _recommended_attrs = { - "http.server.active_requests": _active_requests_count_attrs, - "http.server.duration": _duration_attrs, + "http.server.active_requests": _server_active_requests_count_attrs_old, + "http.server.duration": _server_duration_attrs_old, } @@ -125,7 +125,7 @@ def _test_method(self, method): SpanAttributes.NET_HOST_PORT: 80, SpanAttributes.HTTP_HOST: "falconframework.org", SpanAttributes.HTTP_TARGET: "/", - SpanAttributes.NET_PEER_PORT: "65133", + SpanAttributes.NET_PEER_PORT: 65133, SpanAttributes.HTTP_FLAVOR: "1.1", "falcon.resource": "HelloWorldResource", SpanAttributes.HTTP_STATUS_CODE: 201, @@ -156,7 +156,7 @@ def test_404(self): SpanAttributes.NET_HOST_PORT: 80, SpanAttributes.HTTP_HOST: "falconframework.org", SpanAttributes.HTTP_TARGET: "/", - SpanAttributes.NET_PEER_PORT: "65133", + SpanAttributes.NET_PEER_PORT: 65133, SpanAttributes.HTTP_FLAVOR: "1.1", SpanAttributes.HTTP_STATUS_CODE: 404, }, @@ -193,7 +193,7 @@ def test_500(self): SpanAttributes.NET_HOST_PORT: 80, SpanAttributes.HTTP_HOST: "falconframework.org", SpanAttributes.HTTP_TARGET: "/", - SpanAttributes.NET_PEER_PORT: "65133", + SpanAttributes.NET_PEER_PORT: 65133, SpanAttributes.HTTP_FLAVOR: "1.1", SpanAttributes.HTTP_STATUS_CODE: 500, }, @@ -226,7 +226,7 @@ def test_url_template(self): SpanAttributes.NET_HOST_PORT: 80, SpanAttributes.HTTP_HOST: "falconframework.org", SpanAttributes.HTTP_TARGET: "/", - SpanAttributes.NET_PEER_PORT: "65133", + SpanAttributes.NET_PEER_PORT: 65133, SpanAttributes.HTTP_FLAVOR: "1.1", "falcon.resource": "UserResource", SpanAttributes.HTTP_STATUS_CODE: 200, @@ -336,6 +336,7 @@ def test_falcon_metric_values(self): "http.flavor": "1.1", "http.server_name": "falconframework.org", "net.host.port": 80, + "net.host.name": "falconframework.org", "http.status_code": 404, } expected_requests_count_attributes = { @@ -344,6 +345,8 @@ def test_falcon_metric_values(self): "http.scheme": "http", "http.flavor": "1.1", "http.server_name": "falconframework.org", + "net.host.name": "falconframework.org", + "net.host.port": 80, } start = default_timer() self.client().simulate_get("/hello/756") @@ -523,7 +526,7 @@ def test_custom_request_header_not_added_in_internal_span(self): self.assertNotIn(key, span.attributes) @pytest.mark.skipif( - condition=package_version.parse(_falcon_verison) + condition=package_version.parse(_falcon_version) < package_version.parse("2.0.0"), reason="falcon<2 does not implement custom response headers", ) @@ -558,7 +561,7 @@ def test_custom_response_header_added_in_server_span(self): self.assertNotIn(key, span.attributes) @pytest.mark.skipif( - condition=package_version.parse(_falcon_verison) + condition=package_version.parse(_falcon_version) < package_version.parse("2.0.0"), reason="falcon<2 does not implement custom response headers", ) diff --git a/instrumentation/opentelemetry-instrumentation-flask/tests/test_programmatic.py b/instrumentation/opentelemetry-instrumentation-flask/tests/test_programmatic.py index dec265907f..82ca88460c 100644 --- a/instrumentation/opentelemetry-instrumentation-flask/tests/test_programmatic.py +++ b/instrumentation/opentelemetry-instrumentation-flask/tests/test_programmatic.py @@ -18,17 +18,17 @@ from flask import Flask, request from opentelemetry import trace +from opentelemetry.instrumentation._semconv import ( + _server_active_requests_count_attrs_old, + _server_duration_attrs_old, +) from opentelemetry.instrumentation.flask import FlaskInstrumentor from opentelemetry.instrumentation.propagators import ( TraceResponsePropagator, get_global_response_propagator, set_global_response_propagator, ) -from opentelemetry.instrumentation.wsgi import ( - OpenTelemetryMiddleware, - _active_requests_count_attrs, - _duration_attrs, -) +from opentelemetry.instrumentation.wsgi import OpenTelemetryMiddleware from opentelemetry.sdk.metrics.export import ( HistogramDataPoint, NumberDataPoint, @@ -54,6 +54,7 @@ def expected_attributes(override_attributes): SpanAttributes.HTTP_SERVER_NAME: "localhost", SpanAttributes.HTTP_SCHEME: "http", SpanAttributes.NET_HOST_PORT: 80, + SpanAttributes.NET_HOST_NAME: "localhost", SpanAttributes.HTTP_HOST: "localhost", SpanAttributes.HTTP_TARGET: "/", SpanAttributes.HTTP_FLAVOR: "1.1", @@ -69,8 +70,8 @@ def expected_attributes(override_attributes): "http.server.duration", ] _recommended_attrs = { - "http.server.active_requests": _active_requests_count_attrs, - "http.server.duration": _duration_attrs, + "http.server.active_requests": _server_active_requests_count_attrs_old, + "http.server.duration": _server_duration_attrs_old, } @@ -358,6 +359,7 @@ def test_basic_metric_success(self): "http.server_name": "localhost", "net.host.port": 80, "http.status_code": 200, + "net.host.name": "localhost", } expected_requests_count_attributes = { "http.method": "GET", @@ -365,6 +367,8 @@ def test_basic_metric_success(self): "http.scheme": "http", "http.flavor": "1.1", "http.server_name": "localhost", + "net.host.name": "localhost", + "net.host.port": 80, } self._assert_basic_metric( expected_duration_attributes, @@ -374,20 +378,23 @@ def test_basic_metric_success(self): def test_basic_metric_nonstandard_http_method_success(self): self.client.open("/hello/756", method="NONSTANDARD") expected_duration_attributes = { - "http.method": "UNKNOWN", + "http.method": "_OTHER", "http.host": "localhost", "http.scheme": "http", "http.flavor": "1.1", "http.server_name": "localhost", "net.host.port": 80, "http.status_code": 405, + "net.host.name": "localhost", } expected_requests_count_attributes = { - "http.method": "UNKNOWN", + "http.method": "_OTHER", "http.host": "localhost", "http.scheme": "http", "http.flavor": "1.1", "http.server_name": "localhost", + "net.host.name": "localhost", + "net.host.port": 80, } self._assert_basic_metric( expected_duration_attributes, @@ -410,6 +417,7 @@ def test_basic_metric_nonstandard_http_method_allowed_success(self): "http.server_name": "localhost", "net.host.port": 80, "http.status_code": 405, + "net.host.name": "localhost", } expected_requests_count_attributes = { "http.method": "NONSTANDARD", @@ -417,6 +425,8 @@ def test_basic_metric_nonstandard_http_method_allowed_success(self): "http.scheme": "http", "http.flavor": "1.1", "http.server_name": "localhost", + "net.host.name": "localhost", + "net.host.port": 80, } self._assert_basic_metric( expected_duration_attributes, diff --git a/instrumentation/opentelemetry-instrumentation-grpc/src/opentelemetry/instrumentation/grpc/_aio_client.py b/instrumentation/opentelemetry-instrumentation-grpc/src/opentelemetry/instrumentation/grpc/_aio_client.py index 8fc992be73..9c8cc5cdf3 100644 --- a/instrumentation/opentelemetry-instrumentation-grpc/src/opentelemetry/instrumentation/grpc/_aio_client.py +++ b/instrumentation/opentelemetry-instrumentation-grpc/src/opentelemetry/instrumentation/grpc/_aio_client.py @@ -14,10 +14,9 @@ import functools import logging -from collections import OrderedDict import grpc -from grpc.aio import ClientCallDetails +from grpc.aio import ClientCallDetails, Metadata from opentelemetry.instrumentation.grpc._client import ( OpenTelemetryClientInterceptor, @@ -55,20 +54,19 @@ def callback(call): class _BaseAioClientInterceptor(OpenTelemetryClientInterceptor): @staticmethod - def propagate_trace_in_details(client_call_details): + def propagate_trace_in_details(client_call_details: ClientCallDetails): metadata = client_call_details.metadata if not metadata: - mutable_metadata = OrderedDict() + mutable_metadata = Metadata() else: - mutable_metadata = OrderedDict(metadata) + mutable_metadata = Metadata(*tuple(metadata)) inject(mutable_metadata, setter=_carrier_setter) - metadata = tuple(mutable_metadata.items()) return ClientCallDetails( client_call_details.method, client_call_details.timeout, - metadata, + mutable_metadata, client_call_details.credentials, client_call_details.wait_for_ready, ) diff --git a/instrumentation/opentelemetry-instrumentation-grpc/tests/_aio_client.py b/instrumentation/opentelemetry-instrumentation-grpc/tests/_aio_client.py index 9658df1587..6c0b8eac21 100644 --- a/instrumentation/opentelemetry-instrumentation-grpc/tests/_aio_client.py +++ b/instrumentation/opentelemetry-instrumentation-grpc/tests/_aio_client.py @@ -21,7 +21,7 @@ async def simple_method(stub, error=False): request = Request( client_id=CLIENT_ID, request_data="error" if error else "data" ) - return await stub.SimpleMethod(request) + return await stub.SimpleMethod(request, metadata=(("key", "value"),)) async def client_streaming_method(stub, error=False): @@ -41,7 +41,7 @@ def server_streaming_method(stub, error=False): client_id=CLIENT_ID, request_data="error" if error else "data" ) - return stub.ServerStreamingMethod(request) + return stub.ServerStreamingMethod(request, metadata=(("key", "value"),)) def bidirectional_streaming_method(stub, error=False): @@ -53,4 +53,6 @@ def request_messages(): ) yield request - return stub.BidirectionalStreamingMethod(request_messages()) + return stub.BidirectionalStreamingMethod( + request_messages(), metadata=(("key", "value"),) + ) diff --git a/instrumentation/opentelemetry-instrumentation-grpc/tests/_client.py b/instrumentation/opentelemetry-instrumentation-grpc/tests/_client.py index 69222b37a4..67e7d0a625 100644 --- a/instrumentation/opentelemetry-instrumentation-grpc/tests/_client.py +++ b/instrumentation/opentelemetry-instrumentation-grpc/tests/_client.py @@ -21,14 +21,14 @@ def simple_method(stub, error=False): request = Request( client_id=CLIENT_ID, request_data="error" if error else "data" ) - stub.SimpleMethod(request) + stub.SimpleMethod(request, metadata=(("key", "value"),)) def simple_method_future(stub, error=False): request = Request( client_id=CLIENT_ID, request_data="error" if error else "data" ) - return stub.SimpleMethod.future(request) + return stub.SimpleMethod.future(request, metadata=(("key", "value"),)) def client_streaming_method(stub, error=False): @@ -40,14 +40,18 @@ def request_messages(): ) yield request - stub.ClientStreamingMethod(request_messages()) + stub.ClientStreamingMethod( + request_messages(), metadata=(("key", "value"),) + ) def server_streaming_method(stub, error=False): request = Request( client_id=CLIENT_ID, request_data="error" if error else "data" ) - response_iterator = stub.ServerStreamingMethod(request) + response_iterator = stub.ServerStreamingMethod( + request, metadata=(("key", "value"),) + ) list(response_iterator) @@ -59,6 +63,8 @@ def request_messages(): ) yield request - response_iterator = stub.BidirectionalStreamingMethod(request_messages()) + response_iterator = stub.BidirectionalStreamingMethod( + request_messages(), metadata=(("key", "value"),) + ) list(response_iterator) diff --git a/instrumentation/opentelemetry-instrumentation-grpc/tests/test_aio_client_interceptor.py b/instrumentation/opentelemetry-instrumentation-grpc/tests/test_aio_client_interceptor.py index 6b1006b8a3..21dbc44066 100644 --- a/instrumentation/opentelemetry-instrumentation-grpc/tests/test_aio_client_interceptor.py +++ b/instrumentation/opentelemetry-instrumentation-grpc/tests/test_aio_client_interceptor.py @@ -305,11 +305,10 @@ async def test_client_interceptor_trace_context_propagation(self): await simple_method(stub) metadata = recording_interceptor.recorded_details.metadata - assert len(metadata) == 2 - assert metadata[0][0] == "mock-traceid" - assert metadata[0][1] == "0" - assert metadata[1][0] == "mock-spanid" - assert metadata[1][1] == "0" + assert len(metadata) == 3 + assert metadata.get_all("key") == ["value"] + assert metadata.get_all("mock-traceid") == ["0"] + assert metadata.get_all("mock-spanid") == ["0"] finally: set_global_textmap(previous_propagator) diff --git a/instrumentation/opentelemetry-instrumentation-pyramid/tests/test_automatic.py b/instrumentation/opentelemetry-instrumentation-pyramid/tests/test_automatic.py index 2c3ec85e18..4715e0b461 100644 --- a/instrumentation/opentelemetry-instrumentation-pyramid/tests/test_automatic.py +++ b/instrumentation/opentelemetry-instrumentation-pyramid/tests/test_automatic.py @@ -18,6 +18,10 @@ from pyramid.config import Configurator from opentelemetry import trace +from opentelemetry.instrumentation._semconv import ( + _server_active_requests_count_attrs_old, + _server_duration_attrs_old, +) from opentelemetry.instrumentation.pyramid import PyramidInstrumentor from opentelemetry.sdk.metrics.export import ( HistogramDataPoint, @@ -31,8 +35,6 @@ OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SANITIZE_FIELDS, OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST, OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE, - _active_requests_count_attrs, - _duration_attrs, ) # pylint: disable=import-error @@ -43,8 +45,8 @@ "http.server.duration", ] _recommended_attrs = { - "http.server.active_requests": _active_requests_count_attrs, - "http.server.duration": _duration_attrs, + "http.server.active_requests": _server_active_requests_count_attrs_old, + "http.server.duration": _server_duration_attrs_old, } @@ -213,6 +215,7 @@ def test_basic_metric_success(self): "http.server_name": "localhost", "net.host.port": 80, "http.status_code": 200, + "net.host.name": "localhost", } expected_requests_count_attributes = { "http.method": "GET", @@ -220,6 +223,8 @@ def test_basic_metric_success(self): "http.scheme": "http", "http.flavor": "1.1", "http.server_name": "localhost", + "net.host.name": "localhost", + "net.host.port": 80, } metrics_list = self.memory_metrics_reader.get_metrics_data() for metric in ( diff --git a/instrumentation/opentelemetry-instrumentation-pyramid/tests/test_programmatic.py b/instrumentation/opentelemetry-instrumentation-pyramid/tests/test_programmatic.py index 478eab1937..c566c301d8 100644 --- a/instrumentation/opentelemetry-instrumentation-pyramid/tests/test_programmatic.py +++ b/instrumentation/opentelemetry-instrumentation-pyramid/tests/test_programmatic.py @@ -37,6 +37,7 @@ def expected_attributes(override_attributes): SpanAttributes.HTTP_SERVER_NAME: "localhost", SpanAttributes.HTTP_SCHEME: "http", SpanAttributes.NET_HOST_PORT: 80, + SpanAttributes.NET_HOST_NAME: "localhost", SpanAttributes.HTTP_HOST: "localhost", SpanAttributes.HTTP_TARGET: "/", SpanAttributes.HTTP_FLAVOR: "1.1", diff --git a/instrumentation/opentelemetry-instrumentation-requests/src/opentelemetry/instrumentation/requests/__init__.py b/instrumentation/opentelemetry-instrumentation-requests/src/opentelemetry/instrumentation/requests/__init__.py index d0150d57b7..2052fe47cd 100644 --- a/instrumentation/opentelemetry-instrumentation-requests/src/opentelemetry/instrumentation/requests/__init__.py +++ b/instrumentation/opentelemetry-instrumentation-requests/src/opentelemetry/instrumentation/requests/__init__.py @@ -63,18 +63,20 @@ _SPAN_ATTRIBUTES_ERROR_TYPE, _SPAN_ATTRIBUTES_NETWORK_PEER_ADDRESS, _SPAN_ATTRIBUTES_NETWORK_PEER_PORT, - _filter_duration_attrs, + _client_duration_attrs_new, + _client_duration_attrs_old, + _filter_semconv_duration_attrs, _get_schema_url, + _HTTPStabilityMode, _OpenTelemetrySemanticConventionStability, - _OpenTelemetryStabilityMode, _OpenTelemetryStabilitySignalType, _report_new, _report_old, - _set_http_hostname, + _set_http_host, _set_http_method, - _set_http_net_peer_name, + _set_http_net_peer_name_client, _set_http_network_protocol_version, - _set_http_port, + _set_http_peer_port_client, _set_http_scheme, _set_http_status_code, _set_http_url, @@ -117,7 +119,7 @@ def _instrument( request_hook: _RequestHookT = None, response_hook: _ResponseHookT = None, excluded_urls: ExcludeList = None, - sem_conv_opt_in_mode: _OpenTelemetryStabilityMode = _OpenTelemetryStabilityMode.DEFAULT, + sem_conv_opt_in_mode: _HTTPStabilityMode = _HTTPStabilityMode.DEFAULT, ): """Enables tracing of all requests calls that go through :code:`requests.session.Session.request` (this includes @@ -174,14 +176,14 @@ def get_or_create_headers(): metric_labels, parsed_url.scheme, sem_conv_opt_in_mode ) if parsed_url.hostname: - _set_http_hostname( + _set_http_host( metric_labels, parsed_url.hostname, sem_conv_opt_in_mode ) - _set_http_net_peer_name( + _set_http_net_peer_name_client( metric_labels, parsed_url.hostname, sem_conv_opt_in_mode ) if _report_new(sem_conv_opt_in_mode): - _set_http_hostname( + _set_http_host( span_attributes, parsed_url.hostname, sem_conv_opt_in_mode, @@ -191,11 +193,11 @@ def get_or_create_headers(): _SPAN_ATTRIBUTES_NETWORK_PEER_ADDRESS ] = parsed_url.hostname if parsed_url.port: - _set_http_port( + _set_http_peer_port_client( metric_labels, parsed_url.port, sem_conv_opt_in_mode ) if _report_new(sem_conv_opt_in_mode): - _set_http_port( + _set_http_peer_port_client( span_attributes, parsed_url.port, sem_conv_opt_in_mode ) # Use semconv library when available @@ -284,16 +286,22 @@ def get_or_create_headers(): ).__qualname__ if duration_histogram_old is not None: - duration_attrs_old = _filter_duration_attrs( - metric_labels, _OpenTelemetryStabilityMode.DEFAULT + duration_attrs_old = _filter_semconv_duration_attrs( + metric_labels, + _client_duration_attrs_old, + _client_duration_attrs_new, + _HTTPStabilityMode.DEFAULT, ) duration_histogram_old.record( max(round(elapsed_time * 1000), 0), attributes=duration_attrs_old, ) if duration_histogram_new is not None: - duration_attrs_new = _filter_duration_attrs( - metric_labels, _OpenTelemetryStabilityMode.HTTP + duration_attrs_new = _filter_semconv_duration_attrs( + metric_labels, + _client_duration_attrs_old, + _client_duration_attrs_new, + _HTTPStabilityMode.HTTP, ) duration_histogram_new.record( elapsed_time, attributes=duration_attrs_new @@ -341,7 +349,10 @@ def get_default_span_name(method): Returns: span name """ - return sanitize_method(method.upper().strip()) + method = sanitize_method(method.upper().strip()) + if method == "_OTHER": + return "HTTP" + return method class RequestsInstrumentor(BaseInstrumentor): diff --git a/instrumentation/opentelemetry-instrumentation-requests/src/opentelemetry/instrumentation/requests/package.py b/instrumentation/opentelemetry-instrumentation-requests/src/opentelemetry/instrumentation/requests/package.py index 8424bfeb2a..9cd93a9150 100644 --- a/instrumentation/opentelemetry-instrumentation-requests/src/opentelemetry/instrumentation/requests/package.py +++ b/instrumentation/opentelemetry-instrumentation-requests/src/opentelemetry/instrumentation/requests/package.py @@ -16,3 +16,5 @@ _instruments = ("requests ~= 2.0",) _supports_metrics = True + +_semconv_status = "migration" diff --git a/instrumentation/opentelemetry-instrumentation-requests/tests/test_requests_integration.py b/instrumentation/opentelemetry-instrumentation-requests/tests/test_requests_integration.py index 8817053068..d85d70e20e 100644 --- a/instrumentation/opentelemetry-instrumentation-requests/tests/test_requests_integration.py +++ b/instrumentation/opentelemetry-instrumentation-requests/tests/test_requests_integration.py @@ -23,10 +23,10 @@ import opentelemetry.instrumentation.requests from opentelemetry import trace from opentelemetry.instrumentation._semconv import ( - _OTEL_SEMCONV_STABILITY_OPT_IN_KEY, _SPAN_ATTRIBUTES_ERROR_TYPE, _SPAN_ATTRIBUTES_NETWORK_PEER_ADDRESS, _SPAN_ATTRIBUTES_NETWORK_PEER_PORT, + OTEL_SEMCONV_STABILITY_OPT_IN, _OpenTelemetrySemanticConventionStability, ) from opentelemetry.instrumentation.requests import RequestsInstrumentor @@ -88,7 +88,7 @@ def setUp(self): "os.environ", { "OTEL_PYTHON_REQUESTS_EXCLUDED_URLS": "http://localhost/env_excluded_arg/123,env_excluded_noarg", - _OTEL_SEMCONV_STABILITY_OPT_IN_KEY: sem_conv_mode, + OTEL_SEMCONV_STABILITY_OPT_IN: sem_conv_mode, }, ) @@ -524,7 +524,6 @@ def test_requests_exception_new_semconv(self, *_, **__): self.perform_request(url_with_port) span = self.assert_span() - print(span.attributes) self.assertEqual( span.attributes, { @@ -671,7 +670,7 @@ def setUp(self): self.env_patch = mock.patch.dict( "os.environ", { - _OTEL_SEMCONV_STABILITY_OPT_IN_KEY: sem_conv_mode, + OTEL_SEMCONV_STABILITY_OPT_IN: sem_conv_mode, }, ) self.env_patch.start() diff --git a/instrumentation/opentelemetry-instrumentation-wsgi/src/opentelemetry/instrumentation/wsgi/__init__.py b/instrumentation/opentelemetry-instrumentation-wsgi/src/opentelemetry/instrumentation/wsgi/__init__.py index 50d4f03dff..0a873d0fc3 100644 --- a/instrumentation/opentelemetry-instrumentation-wsgi/src/opentelemetry/instrumentation/wsgi/__init__.py +++ b/instrumentation/opentelemetry-instrumentation-wsgi/src/opentelemetry/instrumentation/wsgi/__init__.py @@ -213,10 +213,34 @@ def response_hook(span: Span, environ: WSGIEnvironment, status: str, response_he from timeit import default_timer from opentelemetry import context, trace -from opentelemetry.instrumentation.utils import ( - _start_internal_or_server_span, - http_status_to_status_code, +from opentelemetry.instrumentation._semconv import ( + _METRIC_ATTRIBUTES_SERVER_DURATION_NAME, + _SPAN_ATTRIBUTES_ERROR_TYPE, + _filter_semconv_active_request_count_attr, + _filter_semconv_duration_attrs, + _get_schema_url, + _HTTPStabilityMode, + _OpenTelemetrySemanticConventionStability, + _OpenTelemetryStabilitySignalType, + _report_new, + _report_old, + _server_active_requests_count_attrs_new, + _server_active_requests_count_attrs_old, + _server_duration_attrs_new, + _server_duration_attrs_old, + _set_http_flavor_version, + _set_http_method, + _set_http_net_host, + _set_http_net_host_port, + _set_http_net_peer_name_server, + _set_http_peer_ip, + _set_http_peer_port_server, + _set_http_scheme, + _set_http_target, + _set_http_user_agent, + _set_status, ) +from opentelemetry.instrumentation.utils import _start_internal_or_server_span from opentelemetry.instrumentation.wsgi.version import __version__ from opentelemetry.metrics import get_meter from opentelemetry.propagators.textmap import Getter @@ -228,6 +252,7 @@ def response_hook(span: Span, environ: WSGIEnvironment, status: str, response_he OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST, OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE, SanitizeValue, + _parse_url_query, get_custom_headers, normalise_request_header_name, normalise_response_header_name, @@ -239,26 +264,6 @@ def response_hook(span: Span, environ: WSGIEnvironment, status: str, response_he _CARRIER_KEY_PREFIX = "HTTP_" _CARRIER_KEY_PREFIX_LEN = len(_CARRIER_KEY_PREFIX) -# List of recommended attributes -_duration_attrs = [ - SpanAttributes.HTTP_METHOD, - SpanAttributes.HTTP_HOST, - SpanAttributes.HTTP_SCHEME, - SpanAttributes.HTTP_STATUS_CODE, - SpanAttributes.HTTP_FLAVOR, - SpanAttributes.HTTP_SERVER_NAME, - SpanAttributes.NET_HOST_NAME, - SpanAttributes.NET_HOST_PORT, -] - -_active_requests_count_attrs = [ - SpanAttributes.HTTP_METHOD, - SpanAttributes.HTTP_HOST, - SpanAttributes.HTTP_SCHEME, - SpanAttributes.HTTP_FLAVOR, - SpanAttributes.HTTP_SERVER_NAME, -] - class WSGIGetter(Getter[dict]): def get( @@ -296,53 +301,84 @@ def setifnotnone(dic, key, value): dic[key] = value -def collect_request_attributes(environ): +# pylint: disable=too-many-branches + + +def collect_request_attributes( + environ, + sem_conv_opt_in_mode=_HTTPStabilityMode.DEFAULT, +): """Collects HTTP request attributes from the PEP3333-conforming WSGI environ and returns a dictionary to be used as span creation attributes. """ + result = {} + _set_http_method( + result, + environ.get("REQUEST_METHOD", ""), + sanitize_method(environ.get("REQUEST_METHOD", "")), + sem_conv_opt_in_mode, + ) + # old semconv v1.12.0 + server_name = environ.get("SERVER_NAME") + if _report_old(sem_conv_opt_in_mode): + result[SpanAttributes.HTTP_SERVER_NAME] = server_name + + _set_http_scheme( + result, + environ.get("wsgi.url_scheme"), + sem_conv_opt_in_mode, + ) - result = { - SpanAttributes.HTTP_METHOD: sanitize_method( - environ.get("REQUEST_METHOD") - ), - SpanAttributes.HTTP_SERVER_NAME: environ.get("SERVER_NAME"), - SpanAttributes.HTTP_SCHEME: environ.get("wsgi.url_scheme"), - } - + host = environ.get("HTTP_HOST") host_port = environ.get("SERVER_PORT") - if host_port is not None and not host_port == "": - result.update({SpanAttributes.NET_HOST_PORT: int(host_port)}) + if host: + _set_http_net_host(result, host, sem_conv_opt_in_mode) + # old semconv v1.12.0 + if _report_old(sem_conv_opt_in_mode): + result[SpanAttributes.HTTP_HOST] = host + if host_port: + _set_http_net_host_port( + result, + int(host_port), + sem_conv_opt_in_mode, + ) - setifnotnone(result, SpanAttributes.HTTP_HOST, environ.get("HTTP_HOST")) target = environ.get("RAW_URI") if target is None: # Note: `"" or None is None` target = environ.get("REQUEST_URI") - if target is not None: - result[SpanAttributes.HTTP_TARGET] = target + if target: + path, query = _parse_url_query(target) + _set_http_target(result, target, path, query, sem_conv_opt_in_mode) else: - result[SpanAttributes.HTTP_URL] = remove_url_credentials( - wsgiref_util.request_uri(environ) - ) + # old semconv v1.20.0 + if _report_old(sem_conv_opt_in_mode): + result[SpanAttributes.HTTP_URL] = remove_url_credentials( + wsgiref_util.request_uri(environ) + ) remote_addr = environ.get("REMOTE_ADDR") if remote_addr: - result[SpanAttributes.NET_PEER_IP] = remote_addr + _set_http_peer_ip(result, remote_addr, sem_conv_opt_in_mode) + + peer_port = environ.get("REMOTE_PORT") + if peer_port: + _set_http_peer_port_server(result, peer_port, sem_conv_opt_in_mode) + remote_host = environ.get("REMOTE_HOST") if remote_host and remote_host != remote_addr: - result[SpanAttributes.NET_PEER_NAME] = remote_host + _set_http_net_peer_name_server( + result, remote_host, sem_conv_opt_in_mode + ) user_agent = environ.get("HTTP_USER_AGENT") if user_agent is not None and len(user_agent) > 0: - result[SpanAttributes.HTTP_USER_AGENT] = user_agent + _set_http_user_agent(result, user_agent, sem_conv_opt_in_mode) - setifnotnone( - result, SpanAttributes.NET_PEER_PORT, environ.get("REMOTE_PORT") - ) flavor = environ.get("SERVER_PROTOCOL", "") if flavor.upper().startswith(_HTTP_VERSION_PREFIX): flavor = flavor[len(_HTTP_VERSION_PREFIX) :] if flavor: - result[SpanAttributes.HTTP_FLAVOR] = flavor + _set_http_flavor_version(result, flavor, sem_conv_opt_in_mode) return result @@ -410,46 +446,56 @@ def _parse_status_code(resp_status): return None -def _parse_active_request_count_attrs(req_attrs): - active_requests_count_attrs = {} - for attr_key in _active_requests_count_attrs: - if req_attrs.get(attr_key) is not None: - active_requests_count_attrs[attr_key] = req_attrs[attr_key] - return active_requests_count_attrs +def _parse_active_request_count_attrs( + req_attrs, sem_conv_opt_in_mode=_HTTPStabilityMode.DEFAULT +): + return _filter_semconv_active_request_count_attr( + req_attrs, + _server_active_requests_count_attrs_old, + _server_active_requests_count_attrs_new, + sem_conv_opt_in_mode, + ) -def _parse_duration_attrs(req_attrs): - duration_attrs = {} - for attr_key in _duration_attrs: - if req_attrs.get(attr_key) is not None: - duration_attrs[attr_key] = req_attrs[attr_key] - return duration_attrs +def _parse_duration_attrs( + req_attrs, sem_conv_opt_in_mode=_HTTPStabilityMode.DEFAULT +): + return _filter_semconv_duration_attrs( + req_attrs, + _server_duration_attrs_old, + _server_duration_attrs_new, + sem_conv_opt_in_mode, + ) def add_response_attributes( - span, start_response_status, response_headers + span, + start_response_status, + response_headers, + duration_attrs=None, + sem_conv_opt_in_mode=_HTTPStabilityMode.DEFAULT, ): # pylint: disable=unused-argument """Adds HTTP response attributes to span using the arguments passed to a PEP3333-conforming start_response callable. """ if not span.is_recording(): return - status_code, _ = start_response_status.split(" ", 1) + status_code_str, _ = start_response_status.split(" ", 1) + status_code = 0 try: - status_code = int(status_code) + status_code = int(status_code_str) except ValueError: - span.set_status( - Status( - StatusCode.ERROR, - "Non-integer HTTP status: " + repr(status_code), - ) - ) - else: - span.set_attribute(SpanAttributes.HTTP_STATUS_CODE, status_code) - span.set_status( - Status(http_status_to_status_code(status_code, server_span=True)) - ) + status_code = -1 + if duration_attrs is None: + duration_attrs = {} + _set_status( + span, + duration_attrs, + status_code_str, + status_code, + sem_conv_opt_in_mode, + ) def get_default_span_name(environ): @@ -464,6 +510,8 @@ def get_default_span_name(environ): The span name. """ method = sanitize_method(environ.get("REQUEST_METHOD", "").strip()) + if method == "_OTHER": + return "HTTP" path = environ.get("PATH_INFO", "").strip() if method and path: return f"{method} {path}" @@ -495,42 +543,66 @@ def __init__( tracer_provider=None, meter_provider=None, ): + # initialize semantic conventions opt-in if needed + _OpenTelemetrySemanticConventionStability._initialize() + sem_conv_opt_in_mode = _OpenTelemetrySemanticConventionStability._get_opentelemetry_stability_opt_in_mode( + _OpenTelemetryStabilitySignalType.HTTP, + ) self.wsgi = wsgi self.tracer = trace.get_tracer( __name__, __version__, tracer_provider, - schema_url="https://opentelemetry.io/schemas/1.11.0", + schema_url=_get_schema_url(sem_conv_opt_in_mode), ) self.meter = get_meter( __name__, __version__, meter_provider, - schema_url="https://opentelemetry.io/schemas/1.11.0", - ) - self.duration_histogram = self.meter.create_histogram( - name=MetricInstruments.HTTP_SERVER_DURATION, - unit="ms", - description="Duration of HTTP client requests.", + schema_url=_get_schema_url(sem_conv_opt_in_mode), ) + self.duration_histogram_old = None + if _report_old(sem_conv_opt_in_mode): + self.duration_histogram_old = self.meter.create_histogram( + name=MetricInstruments.HTTP_SERVER_DURATION, + unit="ms", + description="measures the duration of the inbound HTTP request", + ) + self.duration_histogram_new = None + if _report_new(sem_conv_opt_in_mode): + self.duration_histogram_new = self.meter.create_histogram( + name=_METRIC_ATTRIBUTES_SERVER_DURATION_NAME, + unit="s", + description="measures the duration of the inbound HTTP request", + ) + # We don't need a separate active request counter for old/new semantic conventions + # because the new attributes are a subset of the old attributes self.active_requests_counter = self.meter.create_up_down_counter( name=MetricInstruments.HTTP_SERVER_ACTIVE_REQUESTS, - unit="requests", - description="measures the number of concurrent HTTP requests that are currently in-flight", + unit="{request}", + description="Number of active HTTP server requests.", ) self.request_hook = request_hook self.response_hook = response_hook + self._sem_conv_opt_in_mode = sem_conv_opt_in_mode @staticmethod def _create_start_response( - span, start_response, response_hook, duration_attrs + span, + start_response, + response_hook, + duration_attrs, + sem_conv_opt_in_mode, ): @functools.wraps(start_response) def _start_response(status, response_headers, *args, **kwargs): - add_response_attributes(span, status, response_headers) - status_code = _parse_status_code(status) - if status_code is not None: - duration_attrs[SpanAttributes.HTTP_STATUS_CODE] = status_code + add_response_attributes( + span, + status, + response_headers, + duration_attrs, + sem_conv_opt_in_mode, + ) if span.is_recording() and span.kind == trace.SpanKind.SERVER: custom_attributes = collect_custom_response_headers_attributes( response_headers @@ -551,11 +623,13 @@ def __call__(self, environ, start_response): environ: A WSGI environment. start_response: The WSGI start_response callable. """ - req_attrs = collect_request_attributes(environ) + req_attrs = collect_request_attributes( + environ, self._sem_conv_opt_in_mode + ) active_requests_count_attrs = _parse_active_request_count_attrs( - req_attrs + req_attrs, + self._sem_conv_opt_in_mode, ) - duration_attrs = _parse_duration_attrs(req_attrs) span, token = _start_internal_or_server_span( tracer=self.tracer, @@ -584,20 +658,42 @@ def __call__(self, environ, start_response): try: with trace.use_span(span): start_response = self._create_start_response( - span, start_response, response_hook, duration_attrs + span, + start_response, + response_hook, + req_attrs, + self._sem_conv_opt_in_mode, ) iterable = self.wsgi(environ, start_response) return _end_span_after_iterating(iterable, span, token) except Exception as ex: - if span.is_recording(): + if _report_new(self._sem_conv_opt_in_mode): + req_attrs[_SPAN_ATTRIBUTES_ERROR_TYPE] = type(ex).__qualname__ + if span.is_recording(): + span.set_attribute( + _SPAN_ATTRIBUTES_ERROR_TYPE, type(ex).__qualname__ + ) span.set_status(Status(StatusCode.ERROR, str(ex))) span.end() if token is not None: context.detach(token) raise finally: - duration = max(round((default_timer() - start) * 1000), 0) - self.duration_histogram.record(duration, duration_attrs) + duration_s = default_timer() - start + if self.duration_histogram_old: + duration_attrs_old = _parse_duration_attrs( + req_attrs, _HTTPStabilityMode.DEFAULT + ) + self.duration_histogram_old.record( + max(round(duration_s * 1000), 0), duration_attrs_old + ) + if self.duration_histogram_new: + duration_attrs_new = _parse_duration_attrs( + req_attrs, _HTTPStabilityMode.HTTP + ) + self.duration_histogram_new.record( + max(duration_s, 0), duration_attrs_new + ) self.active_requests_counter.add(-1, active_requests_count_attrs) diff --git a/instrumentation/opentelemetry-instrumentation-wsgi/tests/test_wsgi_middleware.py b/instrumentation/opentelemetry-instrumentation-wsgi/tests/test_wsgi_middleware.py index f74dd67867..985fbe0571 100644 --- a/instrumentation/opentelemetry-instrumentation-wsgi/tests/test_wsgi_middleware.py +++ b/instrumentation/opentelemetry-instrumentation-wsgi/tests/test_wsgi_middleware.py @@ -12,6 +12,8 @@ # See the License for the specific language governing permissions and # limitations under the License. +# pylint: disable=too-many-lines + import sys import unittest import wsgiref.util as wsgiref_util @@ -20,6 +22,15 @@ import opentelemetry.instrumentation.wsgi as otel_wsgi from opentelemetry import trace as trace_api +from opentelemetry.instrumentation._semconv import ( + OTEL_SEMCONV_STABILITY_OPT_IN, + _HTTPStabilityMode, + _OpenTelemetrySemanticConventionStability, + _server_active_requests_count_attrs_new, + _server_active_requests_count_attrs_old, + _server_duration_attrs_new, + _server_duration_attrs_old, +) from opentelemetry.sdk.metrics.export import ( HistogramDataPoint, NumberDataPoint, @@ -127,17 +138,58 @@ def wsgi_with_repeat_custom_response_headers(environ, start_response): return [b"*"] -_expected_metric_names = [ +_expected_metric_names_old = [ "http.server.active_requests", "http.server.duration", ] -_recommended_attrs = { - "http.server.active_requests": otel_wsgi._active_requests_count_attrs, - "http.server.duration": otel_wsgi._duration_attrs, +_expected_metric_names_new = [ + "http.server.active_requests", + "http.server.request.duration", +] +_recommended_metrics_attrs_old = { + "http.server.active_requests": _server_active_requests_count_attrs_old, + "http.server.duration": _server_duration_attrs_old, +} +_recommended_metrics_attrs_new = { + "http.server.active_requests": _server_active_requests_count_attrs_new, + "http.server.request.duration": _server_duration_attrs_new, +} +_server_active_requests_count_attrs_both = ( + _server_active_requests_count_attrs_old +) +_server_active_requests_count_attrs_both.extend( + _server_active_requests_count_attrs_new +) +_recommended_metrics_attrs_both = { + "http.server.active_requests": _server_active_requests_count_attrs_both, + "http.server.duration": _server_duration_attrs_old, + "http.server.request.duration": _server_duration_attrs_new, } class TestWsgiApplication(WsgiTestBase): + def setUp(self): + super().setUp() + + test_name = "" + if hasattr(self, "_testMethodName"): + test_name = self._testMethodName + sem_conv_mode = "default" + if "new_semconv" in test_name: + sem_conv_mode = "http" + elif "both_semconv" in test_name: + sem_conv_mode = "http/dup" + self.env_patch = mock.patch.dict( + "os.environ", + { + OTEL_SEMCONV_STABILITY_OPT_IN: sem_conv_mode, + }, + ) + + _OpenTelemetrySemanticConventionStability._initialized = False + + self.env_patch.start() + def validate_response( self, response, @@ -146,6 +198,8 @@ def validate_response( http_method="GET", span_attributes=None, response_headers=None, + old_sem_conv=True, + new_sem_conv=False, ): while True: try: @@ -171,7 +225,8 @@ def validate_response( self.assertEqual(len(span_list), 1) self.assertEqual(span_list[0].name, span_name) self.assertEqual(span_list[0].kind, trace_api.SpanKind.SERVER) - expected_attributes = { + expected_attributes = {} + expected_attributes_old = { SpanAttributes.HTTP_SERVER_NAME: "127.0.0.1", SpanAttributes.HTTP_SCHEME: "http", SpanAttributes.NET_HOST_PORT: 80, @@ -179,10 +234,27 @@ def validate_response( SpanAttributes.HTTP_FLAVOR: "1.0", SpanAttributes.HTTP_URL: "http://127.0.0.1/", SpanAttributes.HTTP_STATUS_CODE: 200, + SpanAttributes.NET_HOST_NAME: "127.0.0.1", } + expected_attributes_new = { + SpanAttributes.SERVER_PORT: 80, + SpanAttributes.SERVER_ADDRESS: "127.0.0.1", + SpanAttributes.NETWORK_PROTOCOL_VERSION: "1.0", + SpanAttributes.HTTP_RESPONSE_STATUS_CODE: 200, + } + if old_sem_conv: + expected_attributes.update(expected_attributes_old) + if new_sem_conv: + expected_attributes.update(expected_attributes_new) + expected_attributes.update(span_attributes or {}) if http_method is not None: - expected_attributes[SpanAttributes.HTTP_METHOD] = http_method + if old_sem_conv: + expected_attributes[SpanAttributes.HTTP_METHOD] = http_method + if new_sem_conv: + expected_attributes[ + SpanAttributes.HTTP_REQUEST_METHOD + ] = http_method self.assertEqual(span_list[0].attributes, expected_attributes) def test_basic_wsgi_call(self): @@ -190,6 +262,16 @@ def test_basic_wsgi_call(self): response = app(self.environ, self.start_response) self.validate_response(response) + def test_basic_wsgi_call_new_semconv(self): + app = otel_wsgi.OpenTelemetryMiddleware(simple_wsgi) + response = app(self.environ, self.start_response) + self.validate_response(response, old_sem_conv=False, new_sem_conv=True) + + def test_basic_wsgi_call_both_semconv(self): + app = otel_wsgi.OpenTelemetryMiddleware(simple_wsgi) + response = app(self.environ, self.start_response) + self.validate_response(response, old_sem_conv=True, new_sem_conv=True) + def test_hooks(self): hook_headers = ( "hook_attr", @@ -283,7 +365,7 @@ def test_wsgi_metrics(self): for scope_metric in resource_metric.scope_metrics: self.assertTrue(len(scope_metric.metrics) != 0) for metric in scope_metric.metrics: - self.assertIn(metric.name, _expected_metric_names) + self.assertIn(metric.name, _expected_metric_names_old) data_points = list(metric.data.data_points) self.assertEqual(len(data_points), 1) for point in data_points: @@ -294,7 +376,77 @@ def test_wsgi_metrics(self): number_data_point_seen = True for attr in point.attributes: self.assertIn( - attr, _recommended_attrs[metric.name] + attr, + _recommended_metrics_attrs_old[metric.name], + ) + self.assertTrue(number_data_point_seen and histogram_data_point_seen) + + def test_wsgi_metrics_new_semconv(self): + app = otel_wsgi.OpenTelemetryMiddleware(error_wsgi_unhandled) + self.assertRaises(ValueError, app, self.environ, self.start_response) + self.assertRaises(ValueError, app, self.environ, self.start_response) + self.assertRaises(ValueError, app, self.environ, self.start_response) + metrics_list = self.memory_metrics_reader.get_metrics_data() + number_data_point_seen = False + histogram_data_point_seen = False + + self.assertTrue(len(metrics_list.resource_metrics) != 0) + for resource_metric in metrics_list.resource_metrics: + self.assertTrue(len(resource_metric.scope_metrics) != 0) + for scope_metric in resource_metric.scope_metrics: + self.assertTrue(len(scope_metric.metrics) != 0) + for metric in scope_metric.metrics: + self.assertIn(metric.name, _expected_metric_names_new) + data_points = list(metric.data.data_points) + self.assertEqual(len(data_points), 1) + for point in data_points: + if isinstance(point, HistogramDataPoint): + self.assertEqual(point.count, 3) + histogram_data_point_seen = True + if isinstance(point, NumberDataPoint): + number_data_point_seen = True + for attr in point.attributes: + self.assertIn( + attr, + _recommended_metrics_attrs_new[metric.name], + ) + self.assertTrue(number_data_point_seen and histogram_data_point_seen) + + def test_wsgi_metrics_both_semconv(self): + app = otel_wsgi.OpenTelemetryMiddleware(error_wsgi_unhandled) + self.assertRaises(ValueError, app, self.environ, self.start_response) + metrics_list = self.memory_metrics_reader.get_metrics_data() + number_data_point_seen = False + histogram_data_point_seen = False + + self.assertTrue(len(metrics_list.resource_metrics) != 0) + for resource_metric in metrics_list.resource_metrics: + self.assertTrue(len(resource_metric.scope_metrics) != 0) + for scope_metric in resource_metric.scope_metrics: + self.assertTrue(len(scope_metric.metrics) != 0) + for metric in scope_metric.metrics: + if metric.unit == "ms": + self.assertEqual(metric.name, "http.server.duration") + elif metric.unit == "s": + self.assertEqual( + metric.name, "http.server.request.duration" + ) + else: + self.assertEqual( + metric.name, "http.server.active_requests" + ) + data_points = list(metric.data.data_points) + self.assertEqual(len(data_points), 1) + for point in data_points: + if isinstance(point, HistogramDataPoint): + self.assertEqual(point.count, 1) + histogram_data_point_seen = True + if isinstance(point, NumberDataPoint): + number_data_point_seen = True + for attr in point.attributes: + self.assertIn( + attr, + _recommended_metrics_attrs_both[metric.name], ) self.assertTrue(number_data_point_seen and histogram_data_point_seen) @@ -303,7 +455,7 @@ def test_nonstandard_http_method(self): app = otel_wsgi.OpenTelemetryMiddleware(simple_wsgi) response = app(self.environ, self.start_response) self.validate_response( - response, span_name="UNKNOWN /", http_method="UNKNOWN" + response, span_name="HTTP", http_method="_OTHER" ) @mock.patch.dict( @@ -349,34 +501,95 @@ def test_request_attributes(self): SpanAttributes.HTTP_SCHEME: "http", SpanAttributes.HTTP_SERVER_NAME: "127.0.0.1", SpanAttributes.HTTP_FLAVOR: "1.0", + SpanAttributes.NET_HOST_NAME: "127.0.0.1", + }, + ) + + def test_request_attributes_new_semconv(self): + self.environ["QUERY_STRING"] = "foo=bar" + self.environ["REQUEST_URI"] = "http://127.0.0.1/?foo=bar" + + attrs = otel_wsgi.collect_request_attributes( + self.environ, + _HTTPStabilityMode.HTTP, + ) + self.assertDictEqual( + attrs, + { + SpanAttributes.HTTP_REQUEST_METHOD: "GET", + SpanAttributes.SERVER_ADDRESS: "127.0.0.1", + SpanAttributes.SERVER_PORT: 80, + SpanAttributes.NETWORK_PROTOCOL_VERSION: "1.0", + SpanAttributes.URL_PATH: "/", + SpanAttributes.URL_QUERY: "foo=bar", }, ) - def validate_url(self, expected_url, raw=False, has_host=True): + def validate_url( + self, + expected_url, + raw=False, + has_host=True, + old_semconv=True, + new_semconv=False, + ): parts = urlsplit(expected_url) - expected = { + expected_old = { SpanAttributes.HTTP_SCHEME: parts.scheme, SpanAttributes.NET_HOST_PORT: parts.port or (80 if parts.scheme == "http" else 443), SpanAttributes.HTTP_SERVER_NAME: parts.hostname, # Not true in the general case, but for all tests. } - if raw: - expected[SpanAttributes.HTTP_TARGET] = expected_url.split( - parts.netloc, 1 - )[1] - else: - expected[SpanAttributes.HTTP_URL] = expected_url - if has_host: - expected[SpanAttributes.HTTP_HOST] = parts.hostname + expected_new = { + SpanAttributes.SERVER_PORT: parts.port + or (80 if parts.scheme == "http" else 443), + SpanAttributes.SERVER_ADDRESS: parts.hostname, + SpanAttributes.URL_PATH: parts.path, + SpanAttributes.URL_QUERY: parts.query, + } + if old_semconv: + if raw: + expected_old[SpanAttributes.HTTP_TARGET] = expected_url.split( + parts.netloc, 1 + )[1] + else: + expected_old[SpanAttributes.HTTP_URL] = expected_url + if has_host: + expected_old[SpanAttributes.HTTP_HOST] = parts.hostname + if new_semconv: + if raw: + expected_new[SpanAttributes.URL_PATH] = expected_url.split( + parts.path, 1 + )[1] + if parts.query: + expected_new[ + SpanAttributes.URL_QUERY + ] = expected_url.split(parts.query, 1)[1] + else: + expected_new[SpanAttributes.HTTP_URL] = expected_url + if has_host: + expected_new[SpanAttributes.SERVER_ADDRESS] = parts.hostname attrs = otel_wsgi.collect_request_attributes(self.environ) self.assertGreaterEqual( - attrs.items(), expected.items(), expected_url + " expected." + attrs.items(), expected_old.items(), expected_url + " expected." ) def test_request_attributes_with_partial_raw_uri(self): - self.environ["RAW_URI"] = "/#top" - self.validate_url("http://127.0.0.1/#top", raw=True) + self.environ["RAW_URI"] = "/?foo=bar/#top" + self.validate_url("http://127.0.0.1/?foo=bar/#top", raw=True) + self.validate_url( + "http://127.0.0.1/?foo=bar/#top", + raw=True, + old_semconv=False, + new_semconv=True, + ) + self.validate_url( + "http://127.0.0.1/?foo=bar/#top", + raw=True, + old_semconv=True, + new_semconv=True, + ) def test_request_attributes_with_partial_raw_uri_and_nonstandard_port( self, @@ -385,18 +598,68 @@ def test_request_attributes_with_partial_raw_uri_and_nonstandard_port( del self.environ["HTTP_HOST"] self.environ["SERVER_PORT"] = "8080" self.validate_url("http://127.0.0.1:8080/?", raw=True, has_host=False) + self.validate_url( + "http://127.0.0.1:8080/?", + raw=True, + has_host=False, + old_semconv=False, + new_semconv=True, + ) + self.validate_url( + "http://127.0.0.1:8080/?", + raw=True, + has_host=False, + old_semconv=True, + new_semconv=True, + ) def test_https_uri_port(self): del self.environ["HTTP_HOST"] self.environ["SERVER_PORT"] = "443" self.environ["wsgi.url_scheme"] = "https" self.validate_url("https://127.0.0.1/", has_host=False) + self.validate_url( + "https://127.0.0.1/", + has_host=False, + old_semconv=False, + new_semconv=True, + ) + self.validate_url( + "https://127.0.0.1/", + has_host=False, + old_semconv=True, + new_semconv=True, + ) self.environ["SERVER_PORT"] = "8080" self.validate_url("https://127.0.0.1:8080/", has_host=False) + self.validate_url( + "https://127.0.0.1:8080/", + has_host=False, + old_semconv=False, + new_semconv=True, + ) + self.validate_url( + "https://127.0.0.1:8080/", + has_host=False, + old_semconv=True, + new_semconv=True, + ) self.environ["SERVER_PORT"] = "80" self.validate_url("https://127.0.0.1:80/", has_host=False) + self.validate_url( + "https://127.0.0.1:80/", + has_host=False, + old_semconv=False, + new_semconv=True, + ) + self.validate_url( + "https://127.0.0.1:80/", + has_host=False, + old_semconv=True, + new_semconv=True, + ) def test_http_uri_port(self): del self.environ["HTTP_HOST"] @@ -438,10 +701,10 @@ def test_request_attributes_with_faux_scheme_relative_raw_uri(self): def test_request_attributes_pathless(self): self.environ["RAW_URI"] = "" - expected = {SpanAttributes.HTTP_TARGET: ""} - self.assertGreaterEqual( - otel_wsgi.collect_request_attributes(self.environ).items(), - expected.items(), + self.assertIsNone( + otel_wsgi.collect_request_attributes(self.environ).get( + SpanAttributes.HTTP_TARGET + ) ) def test_request_attributes_with_full_request_uri(self): @@ -449,29 +712,58 @@ def test_request_attributes_with_full_request_uri(self): self.environ["REQUEST_METHOD"] = "CONNECT" self.environ[ "REQUEST_URI" - ] = "127.0.0.1:8080" # Might happen in a CONNECT request - expected = { + ] = "http://docs.python.org:80/3/library/urllib.parse.html?highlight=params#url-parsing" # Might happen in a CONNECT request + expected_old = { SpanAttributes.HTTP_HOST: "127.0.0.1:8080", - SpanAttributes.HTTP_TARGET: "127.0.0.1:8080", + SpanAttributes.HTTP_TARGET: "http://docs.python.org:80/3/library/urllib.parse.html?highlight=params#url-parsing", + } + expected_new = { + SpanAttributes.URL_PATH: "/3/library/urllib.parse.html", + SpanAttributes.URL_QUERY: "highlight=params", } self.assertGreaterEqual( otel_wsgi.collect_request_attributes(self.environ).items(), - expected.items(), + expected_old.items(), + ) + self.assertGreaterEqual( + otel_wsgi.collect_request_attributes( + self.environ, + _HTTPStabilityMode.HTTP, + ).items(), + expected_new.items(), ) def test_http_user_agent_attribute(self): self.environ["HTTP_USER_AGENT"] = "test-useragent" expected = {SpanAttributes.HTTP_USER_AGENT: "test-useragent"} + expected_new = {SpanAttributes.USER_AGENT_ORIGINAL: "test-useragent"} self.assertGreaterEqual( otel_wsgi.collect_request_attributes(self.environ).items(), expected.items(), ) + self.assertGreaterEqual( + otel_wsgi.collect_request_attributes( + self.environ, + _HTTPStabilityMode.HTTP, + ).items(), + expected_new.items(), + ) def test_response_attributes(self): otel_wsgi.add_response_attributes(self.span, "404 Not Found", {}) + otel_wsgi.add_response_attributes( + self.span, + "404 Not Found", + {}, + sem_conv_opt_in_mode=_HTTPStabilityMode.HTTP, + ) expected = (mock.call(SpanAttributes.HTTP_STATUS_CODE, 404),) - self.assertEqual(self.span.set_attribute.call_count, len(expected)) + expected_new = ( + mock.call(SpanAttributes.HTTP_RESPONSE_STATUS_CODE, 404), + ) + self.assertEqual(self.span.set_attribute.call_count, 2) self.span.set_attribute.assert_has_calls(expected, any_order=True) + self.span.set_attribute.assert_has_calls(expected_new, any_order=True) def test_credential_removal(self): self.environ["HTTP_HOST"] = "username:password@mock" diff --git a/opentelemetry-instrumentation/src/opentelemetry/instrumentation/_semconv.py b/opentelemetry-instrumentation/src/opentelemetry/instrumentation/_semconv.py index fbfc92cf21..31c2486acc 100644 --- a/opentelemetry-instrumentation/src/opentelemetry/instrumentation/_semconv.py +++ b/opentelemetry-instrumentation/src/opentelemetry/instrumentation/_semconv.py @@ -16,13 +16,16 @@ import threading from enum import Enum +from opentelemetry.instrumentation.utils import http_status_to_status_code from opentelemetry.semconv.trace import SpanAttributes +from opentelemetry.trace.status import Status, StatusCode # TODO: will come through semconv package once updated _SPAN_ATTRIBUTES_ERROR_TYPE = "error.type" _SPAN_ATTRIBUTES_NETWORK_PEER_ADDRESS = "network.peer.address" _SPAN_ATTRIBUTES_NETWORK_PEER_PORT = "network.peer.port" _METRIC_ATTRIBUTES_CLIENT_DURATION_NAME = "http.client.request.duration" +_METRIC_ATTRIBUTES_SERVER_DURATION_NAME = "http.server.request.duration" _client_duration_attrs_old = [ SpanAttributes.HTTP_STATUS_CODE, @@ -45,13 +48,117 @@ # SpanAttributes.URL_SCHEME, ] +_server_duration_attrs_old = [ + SpanAttributes.HTTP_METHOD, + SpanAttributes.HTTP_HOST, + SpanAttributes.HTTP_SCHEME, + SpanAttributes.HTTP_STATUS_CODE, + SpanAttributes.HTTP_FLAVOR, + SpanAttributes.HTTP_SERVER_NAME, + SpanAttributes.NET_HOST_NAME, + SpanAttributes.NET_HOST_PORT, +] + +_server_duration_attrs_new = [ + _SPAN_ATTRIBUTES_ERROR_TYPE, + SpanAttributes.HTTP_REQUEST_METHOD, + SpanAttributes.HTTP_RESPONSE_STATUS_CODE, + SpanAttributes.HTTP_ROUTE, + SpanAttributes.NETWORK_PROTOCOL_VERSION, + # TODO: Support opt-in for scheme in new semconv + # SpanAttributes.URL_SCHEME, +] + +_server_active_requests_count_attrs_old = [ + SpanAttributes.HTTP_METHOD, + SpanAttributes.HTTP_HOST, + SpanAttributes.HTTP_SCHEME, + SpanAttributes.HTTP_FLAVOR, + SpanAttributes.HTTP_SERVER_NAME, + SpanAttributes.NET_HOST_NAME, + SpanAttributes.NET_HOST_PORT, +] + +_server_active_requests_count_attrs_new = [ + SpanAttributes.HTTP_REQUEST_METHOD, + SpanAttributes.URL_SCHEME, +] + +OTEL_SEMCONV_STABILITY_OPT_IN = "OTEL_SEMCONV_STABILITY_OPT_IN" + + +class _OpenTelemetryStabilitySignalType: + HTTP = "http" + + +class _HTTPStabilityMode(Enum): + # http - emit the new, stable HTTP and networking conventions ONLY + HTTP = "http" + # http/dup - emit both the old and the stable HTTP and networking conventions + HTTP_DUP = "http/dup" + # default - continue emitting old experimental HTTP and networking conventions + DEFAULT = "default" + + +def _report_new(mode): + return mode.name != _HTTPStabilityMode.DEFAULT.name + + +def _report_old(mode): + return mode.name != _HTTPStabilityMode.HTTP.name -def _filter_duration_attrs(attrs, sem_conv_opt_in_mode): + +class _OpenTelemetrySemanticConventionStability: + _initialized = False + _lock = threading.Lock() + _OTEL_SEMCONV_STABILITY_SIGNAL_MAPPING = {} + + @classmethod + def _initialize(cls): + with _OpenTelemetrySemanticConventionStability._lock: + if not _OpenTelemetrySemanticConventionStability._initialized: + # Users can pass in comma delimited string for opt-in options + # Only values for http stability are supported for now + opt_in = os.environ.get(OTEL_SEMCONV_STABILITY_OPT_IN, "") + opt_in_list = [] + if opt_in: + opt_in_list = [s.strip() for s in opt_in.split(",")] + http_opt_in = _HTTPStabilityMode.DEFAULT + if opt_in_list: + # Process http opt-in + # http/dup takes priority over http + if _HTTPStabilityMode.HTTP_DUP.value in opt_in_list: + http_opt_in = _HTTPStabilityMode.HTTP_DUP + elif _HTTPStabilityMode.HTTP.value in opt_in_list: + http_opt_in = _HTTPStabilityMode.HTTP + _OpenTelemetrySemanticConventionStability._OTEL_SEMCONV_STABILITY_SIGNAL_MAPPING[ + _OpenTelemetryStabilitySignalType.HTTP + ] = http_opt_in + _OpenTelemetrySemanticConventionStability._initialized = True + + @classmethod + # Get OpenTelemetry opt-in mode based off of signal type (http, messaging, etc.) + def _get_opentelemetry_stability_opt_in_mode( + cls, + signal_type: _OpenTelemetryStabilitySignalType, + ) -> _HTTPStabilityMode: + return _OpenTelemetrySemanticConventionStability._OTEL_SEMCONV_STABILITY_SIGNAL_MAPPING.get( + signal_type, _HTTPStabilityMode.DEFAULT + ) + + +def _filter_semconv_duration_attrs( + attrs, + old_attrs, + new_attrs, + sem_conv_opt_in_mode=_HTTPStabilityMode.DEFAULT, +): filtered_attrs = {} + # duration is two different metrics depending on sem_conv_opt_in_mode, so no DUP attributes allowed_attributes = ( - _client_duration_attrs_new - if sem_conv_opt_in_mode == _OpenTelemetryStabilityMode.HTTP - else _client_duration_attrs_old + new_attrs + if sem_conv_opt_in_mode == _HTTPStabilityMode.HTTP + else old_attrs ) for key, val in attrs.items(): if key in allowed_attributes: @@ -59,6 +166,24 @@ def _filter_duration_attrs(attrs, sem_conv_opt_in_mode): return filtered_attrs +def _filter_semconv_active_request_count_attr( + attrs, + old_attrs, + new_attrs, + sem_conv_opt_in_mode=_HTTPStabilityMode.DEFAULT, +): + filtered_attrs = {} + if _report_old(sem_conv_opt_in_mode): + for key, val in attrs.items(): + if key in old_attrs: + filtered_attrs[key] = val + if _report_new(sem_conv_opt_in_mode): + for key, val in attrs.items(): + if key in new_attrs: + filtered_attrs[key] = val + return filtered_attrs + + def set_string_attribute(result, key, value): if value: result[key] = value @@ -90,6 +215,15 @@ def _set_http_method(result, original, normalized, sem_conv_opt_in_mode): ) +def _set_http_status_code(result, code, sem_conv_opt_in_mode): + if _report_old(sem_conv_opt_in_mode): + set_int_attribute(result, SpanAttributes.HTTP_STATUS_CODE, code) + if _report_new(sem_conv_opt_in_mode): + set_int_attribute( + result, SpanAttributes.HTTP_RESPONSE_STATUS_CODE, code + ) + + def _set_http_url(result, url, sem_conv_opt_in_mode): if _report_old(sem_conv_opt_in_mode): set_string_attribute(result, SpanAttributes.HTTP_URL, url) @@ -105,36 +239,30 @@ def _set_http_scheme(result, scheme, sem_conv_opt_in_mode): # set_string_attribute(result, SpanAttributes.URL_SCHEME, scheme) -def _set_http_hostname(result, hostname, sem_conv_opt_in_mode): +def _set_http_host(result, host, sem_conv_opt_in_mode): if _report_old(sem_conv_opt_in_mode): - set_string_attribute(result, SpanAttributes.HTTP_HOST, hostname) + set_string_attribute(result, SpanAttributes.HTTP_HOST, host) if _report_new(sem_conv_opt_in_mode): - set_string_attribute(result, SpanAttributes.SERVER_ADDRESS, hostname) + set_string_attribute(result, SpanAttributes.SERVER_ADDRESS, host) + +# Client -def _set_http_net_peer_name(result, peer_name, sem_conv_opt_in_mode): + +def _set_http_net_peer_name_client(result, peer_name, sem_conv_opt_in_mode): if _report_old(sem_conv_opt_in_mode): set_string_attribute(result, SpanAttributes.NET_PEER_NAME, peer_name) if _report_new(sem_conv_opt_in_mode): set_string_attribute(result, SpanAttributes.SERVER_ADDRESS, peer_name) -def _set_http_port(result, port, sem_conv_opt_in_mode): +def _set_http_peer_port_client(result, port, sem_conv_opt_in_mode): if _report_old(sem_conv_opt_in_mode): set_int_attribute(result, SpanAttributes.NET_PEER_PORT, port) if _report_new(sem_conv_opt_in_mode): set_int_attribute(result, SpanAttributes.SERVER_PORT, port) -def _set_http_status_code(result, code, sem_conv_opt_in_mode): - if _report_old(sem_conv_opt_in_mode): - set_int_attribute(result, SpanAttributes.HTTP_STATUS_CODE, code) - if _report_new(sem_conv_opt_in_mode): - set_int_attribute( - result, SpanAttributes.HTTP_RESPONSE_STATUS_CODE, code - ) - - def _set_http_network_protocol_version(result, version, sem_conv_opt_in_mode): if _report_old(sem_conv_opt_in_mode): set_string_attribute(result, SpanAttributes.HTTP_FLAVOR, version) @@ -144,74 +272,117 @@ def _set_http_network_protocol_version(result, version, sem_conv_opt_in_mode): ) -_OTEL_SEMCONV_STABILITY_OPT_IN_KEY = "OTEL_SEMCONV_STABILITY_OPT_IN" +# Server -class _OpenTelemetryStabilitySignalType: - HTTP = "http" +def _set_http_net_host(result, host, sem_conv_opt_in_mode): + if _report_old(sem_conv_opt_in_mode): + set_string_attribute(result, SpanAttributes.NET_HOST_NAME, host) + if _report_new(sem_conv_opt_in_mode): + set_string_attribute(result, SpanAttributes.SERVER_ADDRESS, host) -class _OpenTelemetryStabilityMode(Enum): - # http - emit the new, stable HTTP and networking conventions ONLY - HTTP = "http" - # http/dup - emit both the old and the stable HTTP and networking conventions - HTTP_DUP = "http/dup" - # default - continue emitting old experimental HTTP and networking conventions - DEFAULT = "default" +def _set_http_net_host_port(result, port, sem_conv_opt_in_mode): + if _report_old(sem_conv_opt_in_mode): + set_int_attribute(result, SpanAttributes.NET_HOST_PORT, port) + if _report_new(sem_conv_opt_in_mode): + set_int_attribute(result, SpanAttributes.SERVER_PORT, port) -def _report_new(mode): - return mode.name != _OpenTelemetryStabilityMode.DEFAULT.name +def _set_http_target(result, target, path, query, sem_conv_opt_in_mode): + if _report_old(sem_conv_opt_in_mode): + set_string_attribute(result, SpanAttributes.HTTP_TARGET, target) + if _report_new(sem_conv_opt_in_mode): + if path: + set_string_attribute(result, SpanAttributes.URL_PATH, path) + if query: + set_string_attribute(result, SpanAttributes.URL_QUERY, query) -def _report_old(mode): - return mode.name != _OpenTelemetryStabilityMode.HTTP.name +def _set_http_peer_ip(result, ip, sem_conv_opt_in_mode): + if _report_old(sem_conv_opt_in_mode): + set_string_attribute(result, SpanAttributes.NET_PEER_IP, ip) + if _report_new(sem_conv_opt_in_mode): + set_string_attribute(result, SpanAttributes.CLIENT_ADDRESS, ip) -class _OpenTelemetrySemanticConventionStability: - _initialized = False - _lock = threading.Lock() - _OTEL_SEMCONV_STABILITY_SIGNAL_MAPPING = {} +def _set_http_peer_port_server(result, port, sem_conv_opt_in_mode): + if _report_old(sem_conv_opt_in_mode): + set_int_attribute(result, SpanAttributes.NET_PEER_PORT, port) + if _report_new(sem_conv_opt_in_mode): + set_int_attribute(result, SpanAttributes.CLIENT_PORT, port) - @classmethod - def _initialize(cls): - with _OpenTelemetrySemanticConventionStability._lock: - if not _OpenTelemetrySemanticConventionStability._initialized: - # Users can pass in comma delimited string for opt-in options - # Only values for http stability are supported for now - opt_in = os.environ.get(_OTEL_SEMCONV_STABILITY_OPT_IN_KEY, "") - opt_in_list = [] - if opt_in: - opt_in_list = [s.strip() for s in opt_in.split(",")] - http_opt_in = _OpenTelemetryStabilityMode.DEFAULT - if opt_in_list: - # Process http opt-in - # http/dup takes priority over http - if ( - _OpenTelemetryStabilityMode.HTTP_DUP.value - in opt_in_list - ): - http_opt_in = _OpenTelemetryStabilityMode.HTTP_DUP - elif _OpenTelemetryStabilityMode.HTTP.value in opt_in_list: - http_opt_in = _OpenTelemetryStabilityMode.HTTP - _OpenTelemetrySemanticConventionStability._OTEL_SEMCONV_STABILITY_SIGNAL_MAPPING[ - _OpenTelemetryStabilitySignalType.HTTP - ] = http_opt_in - _OpenTelemetrySemanticConventionStability._initialized = True - @classmethod - # Get OpenTelemetry opt-in mode based off of signal type (http, messaging, etc.) - def _get_opentelemetry_stability_opt_in_mode( - cls, - signal_type: _OpenTelemetryStabilitySignalType, - ) -> _OpenTelemetryStabilityMode: - return _OpenTelemetrySemanticConventionStability._OTEL_SEMCONV_STABILITY_SIGNAL_MAPPING.get( - signal_type, _OpenTelemetryStabilityMode.DEFAULT +def _set_http_user_agent(result, user_agent, sem_conv_opt_in_mode): + if _report_old(sem_conv_opt_in_mode): + set_string_attribute( + result, SpanAttributes.HTTP_USER_AGENT, user_agent + ) + if _report_new(sem_conv_opt_in_mode): + set_string_attribute( + result, SpanAttributes.USER_AGENT_ORIGINAL, user_agent + ) + + +def _set_http_net_peer_name_server(result, name, sem_conv_opt_in_mode): + if _report_old(sem_conv_opt_in_mode): + set_string_attribute(result, SpanAttributes.NET_PEER_NAME, name) + if _report_new(sem_conv_opt_in_mode): + set_string_attribute(result, SpanAttributes.CLIENT_ADDRESS, name) + + +def _set_http_flavor_version(result, version, sem_conv_opt_in_mode): + if _report_old(sem_conv_opt_in_mode): + set_string_attribute(result, SpanAttributes.HTTP_FLAVOR, version) + if _report_new(sem_conv_opt_in_mode): + set_string_attribute( + result, SpanAttributes.NETWORK_PROTOCOL_VERSION, version + ) + + +def _set_status( + span, + metrics_attributes, + status_code_str, + status_code, + sem_conv_opt_in_mode, +): + if status_code < 0: + if _report_new(sem_conv_opt_in_mode): + span.set_attribute(_SPAN_ATTRIBUTES_ERROR_TYPE, status_code_str) + metrics_attributes[_SPAN_ATTRIBUTES_ERROR_TYPE] = status_code_str + + span.set_status( + Status( + StatusCode.ERROR, + "Non-integer HTTP status: " + status_code_str, + ) ) + else: + status = http_status_to_status_code(status_code, server_span=True) + + if _report_old(sem_conv_opt_in_mode): + span.set_attribute(SpanAttributes.HTTP_STATUS_CODE, status_code) + metrics_attributes[SpanAttributes.HTTP_STATUS_CODE] = status_code + if _report_new(sem_conv_opt_in_mode): + span.set_attribute( + SpanAttributes.HTTP_RESPONSE_STATUS_CODE, status_code + ) + metrics_attributes[ + SpanAttributes.HTTP_RESPONSE_STATUS_CODE + ] = status_code + if status == StatusCode.ERROR: + span.set_attribute( + _SPAN_ATTRIBUTES_ERROR_TYPE, status_code_str + ) + metrics_attributes[ + _SPAN_ATTRIBUTES_ERROR_TYPE + ] = status_code_str + span.set_status(Status(status)) # Get schema version based off of opt-in mode -def _get_schema_url(mode: _OpenTelemetryStabilityMode) -> str: - if mode is _OpenTelemetryStabilityMode.DEFAULT: +def _get_schema_url(mode: _HTTPStabilityMode) -> str: + if mode is _HTTPStabilityMode.DEFAULT: return "https://opentelemetry.io/schemas/1.11.0" return SpanAttributes.SCHEMA_URL diff --git a/opentelemetry-instrumentation/src/opentelemetry/instrumentation/bootstrap_gen.py b/opentelemetry-instrumentation/src/opentelemetry/instrumentation/bootstrap_gen.py index 455d08e41a..9eebd5bb38 100644 --- a/opentelemetry-instrumentation/src/opentelemetry/instrumentation/bootstrap_gen.py +++ b/opentelemetry-instrumentation/src/opentelemetry/instrumentation/bootstrap_gen.py @@ -73,7 +73,7 @@ "instrumentation": "opentelemetry-instrumentation-django==0.46b0.dev", }, { - "library": "elasticsearch >= 2.0", + "library": "elasticsearch >= 6.0", "instrumentation": "opentelemetry-instrumentation-elasticsearch==0.46b0.dev", }, { diff --git a/scripts/build.sh b/scripts/build.sh index fa490a6a35..247bb24b4e 100755 --- a/scripts/build.sh +++ b/scripts/build.sh @@ -8,13 +8,13 @@ set -ev # Get the latest versions of packaging tools python3 -m pip install --upgrade pip build setuptools wheel -BASEDIR=$(dirname $(readlink -f $(dirname $0))) +BASEDIR=$(dirname "$(readlink -f "$(dirname $0)")") DISTDIR=dist ( cd $BASEDIR mkdir -p $DISTDIR - rm -rf $DISTDIR/* + rm -rf ${DISTDIR:?}/* for d in exporter/*/ opentelemetry-instrumentation/ opentelemetry-contrib-instrumentations/ opentelemetry-distro/ instrumentation/*/ propagator/*/ resource/*/ sdk-extension/*/ util/*/ ; do ( diff --git a/scripts/build_a_package.sh b/scripts/build_a_package.sh index f3baa8a65a..333deccd80 100755 --- a/scripts/build_a_package.sh +++ b/scripts/build_a_package.sh @@ -22,7 +22,7 @@ set -ev if [ -z $GITHUB_REF ]; then echo 'Failed to run script, missing workflow env variable GITHUB_REF.' - exit -1 + exit 1 fi pkg_name_and_version=${GITHUB_REF#refs/tags/*} @@ -40,13 +40,13 @@ cd $basedir distdir=${basedir}/dist mkdir -p $distdir -rm -rf $distdir/* +rm -rf ${distdir:?}/* pyproject_toml_file_path=$(ls **/$pkg_name/pyproject.toml) if [ -z $pyproject_toml_file_path ]; then echo "Error! pyproject.toml not found for $pkg_name, can't build." - exit -1 + exit 1 fi directory_with_package=$(dirname $pyproject_toml_file_path) @@ -61,7 +61,7 @@ pkg_tar_gz_file=${pkg_name}-${pkg_version}.tar.gz if ! [ -f $pkg_tar_gz_file ]; then echo 'Error! Tag version does not match version built using latest package files.' - exit -1 + exit 1 fi # Build a wheel for the source distribution diff --git a/scripts/coverage.sh b/scripts/coverage.sh index 4015c6884a..02ce1eed55 100755 --- a/scripts/coverage.sh +++ b/scripts/coverage.sh @@ -11,9 +11,6 @@ function cov { ${1} } -PYTHON_VERSION=$(python -c 'import sys; print(".".join(map(str, sys.version_info[:3])))') -PYTHON_VERSION_INFO=(${PYTHON_VERSION//./ }) - coverage erase cov instrumentation/opentelemetry-instrumentation-flask diff --git a/scripts/generate_instrumentation_readme.py b/scripts/generate_instrumentation_readme.py index 00045cd73c..79b8a5533d 100755 --- a/scripts/generate_instrumentation_readme.py +++ b/scripts/generate_instrumentation_readme.py @@ -23,8 +23,8 @@ _prefix = "opentelemetry-instrumentation-" header = """ -| Instrumentation | Supported Packages | Metrics support | -| --------------- | ------------------ | --------------- |""" +| Instrumentation | Supported Packages | Metrics support | Semconv status | +| --------------- | ------------------ | --------------- | -------------- |""" def main(): @@ -63,13 +63,17 @@ def main(): instruments = pkg_info["_instruments"] supports_metrics = pkg_info.get("_supports_metrics") + semconv_status = pkg_info.get("_semconv_status") if not instruments: instruments = (name,) + if not semconv_status: + semconv_status = "experimental" + metric_column = "Yes" if supports_metrics else "No" table.append( - f"| [{instrumentation}](./{instrumentation}) | {','.join(instruments)} | {metric_column}" + f"| [{instrumentation}](./{instrumentation}) | {','.join(instruments)} | {metric_column} | {semconv_status}" ) with open( diff --git a/tox.ini b/tox.ini index 420ae899f0..bee544cb70 100644 --- a/tox.ini +++ b/tox.ini @@ -27,7 +27,7 @@ envlist = pypy3-test-instrumentation-aiohttp-client ; opentelemetry-instrumentation-aiohttp-server - py3{6,8,9,10,11}-test-instrumentation-aiohttp-server + py3{8,9,10,11}-test-instrumentation-aiohttp-server pypy3-test-instrumentation-aiohttp-server ; opentelemetry-instrumentation-aiopg @@ -44,7 +44,7 @@ envlist = ; pypy3-test-instrumentation-botocore ; opentelemetry-instrumentation-boto3sqs - py3{6,8,9,10,11}-test-instrumentation-boto3sqs + py3{8,9,10,11}-test-instrumentation-boto3sqs ; FIXME: see https://github.com/open-telemetry/opentelemetry-python-contrib/issues/1736 ; pypy3-test-instrumentation-boto3sqs @@ -78,13 +78,9 @@ envlist = ; FIXME: Elasticsearch >=7 causes CI workflow tests to hang, see open-telemetry/opentelemetry-python-contrib#620 ; The numbers at the end of the environment names ; below mean these dependencies are being used: - ; 0: elasticsearch-dsl>=2.0,<3.0 elasticsearch>=2.0,<3.0 - ; 1: elasticsearch-dsl>=5.0,<6.0 elasticsearch>=5.0,<6.0 - ; 2: elasticsearch-dsl>=6.0,<7.0 elasticsearch>=6.0,<7.0 - py3{8,9,10,11}-test-instrumentation-elasticsearch-{0,2} - pypy3-test-instrumentation-elasticsearch-{0,2} - py3{8,9}-test-instrumentation-elasticsearch-1 - pypy3-test-instrumentation-elasticsearch-1 + ; 6: elasticsearch-dsl>=6.0,<7.0 elasticsearch>=6.0,<7.0 + py3{8,9,10,11}-test-instrumentation-elasticsearch-6 + pypy3-test-instrumentation-elasticsearch-6 ; opentelemetry-instrumentation-falcon ; py310 does not work with falcon 1 @@ -144,7 +140,7 @@ envlist = pypy3-test-exporter-richconsole ; opentelemetry-exporter-prometheus-remote-write - py3{6,8,9,10,11}-test-exporter-prometheus-remote-write + py3{8,9,10,11}-test-exporter-prometheus-remote-write pypy3-test-exporter-prometheus-remote-write ; opentelemetry-instrumentation-mysql @@ -160,7 +156,7 @@ envlist = ; ext-psycopg2 intentionally excluded from pypy3 ; opentelemetry-instrumentation-psycopg - py3{7,8,9,10,11}-test-instrumentation-psycopg + py3{8,9,10,11}-test-instrumentation-psycopg pypy3-test-instrumentation-psycopg ; opentelemetry-instrumentation-pymemcache @@ -230,7 +226,7 @@ envlist = py3{8}-test-instrumentation-sklearn ; opentelemetry-instrumentation-system-metrics - py3{6,8,9,10,11}-test-instrumentation-system-metrics + py3{8,9,10,11}-test-instrumentation-system-metrics pypy3-test-instrumentation-system-metrics ; opentelemetry-instrumentation-threading @@ -452,9 +448,7 @@ commands_pre = sqlalchemy-0: pip install -r {toxinidir}/instrumentation/opentelemetry-instrumentation-sqlalchemy/test-requirements-0.txt sqlalchemy-1: pip install -r {toxinidir}/instrumentation/opentelemetry-instrumentation-sqlalchemy/test-requirements-1.txt - elasticsearch-0: pip install -r {toxinidir}/instrumentation/opentelemetry-instrumentation-elasticsearch/test-requirements-0.txt - elasticsearch-1: pip install -r {toxinidir}/instrumentation/opentelemetry-instrumentation-elasticsearch/test-requirements-1.txt - elasticsearch-2: pip install -r {toxinidir}/instrumentation/opentelemetry-instrumentation-elasticsearch/test-requirements-2.txt + elasticsearch-6: pip install -r {toxinidir}/instrumentation/opentelemetry-instrumentation-elasticsearch/test-requirements-6.txt asyncio: pip install -r {toxinidir}/instrumentation/opentelemetry-instrumentation-asyncio/test-requirements.txt @@ -617,7 +611,7 @@ commands_pre = # prerequisite: follow the instructions here https://github.com/PyMySQL/mysqlclient#install # for your OS to install the required dependencies pip install -r {toxinidir}/instrumentation/opentelemetry-instrumentation-mysqlclient/test-requirements.txt - pip install -r {toxinidir}/instrumentation/opentelemetry-instrumentation-elasticsearch/test-requirements-2.txt + pip install -r {toxinidir}/instrumentation/opentelemetry-instrumentation-elasticsearch/test-requirements-6.txt pip install -r {toxinidir}/instrumentation/opentelemetry-instrumentation-tornado/test-requirements.txt pip install -r {toxinidir}/instrumentation/opentelemetry-instrumentation-asyncpg/test-requirements.txt pip install -r {toxinidir}/instrumentation/opentelemetry-instrumentation-pymongo/test-requirements.txt diff --git a/util/opentelemetry-util-http/src/opentelemetry/util/http/__init__.py b/util/opentelemetry-util-http/src/opentelemetry/util/http/__init__.py index 523f9400b1..1f7ce98937 100644 --- a/util/opentelemetry-util-http/src/opentelemetry/util/http/__init__.py +++ b/util/opentelemetry-util-http/src/opentelemetry/util/http/__init__.py @@ -218,7 +218,7 @@ def sanitize_method(method: Optional[str]) -> Optional[str]: ] ): return method - return "UNKNOWN" + return "_OTHER" def get_custom_headers(env_var: str) -> list[str]: @@ -245,3 +245,10 @@ def _parse_duration_attrs(req_attrs): for key in _duration_attrs.intersection(req_attrs.keys()) } return duration_attrs + + +def _parse_url_query(url: str): + parsed_url = urlparse(url) + path = parsed_url.path + query_params = parsed_url.query + return path, query_params