Python OpenTelemetry workshop

Course

This lesson is a part of our OpenTelemetry masterclass. If you haven't already, checkout the chapter introduction.

Each lesson in this lab builds on the last one, so make sure you read the workshop introduction before proceeding with this one.

In this workshop, you instrument a Python web application with OpenTelemetry using the fundamentals you learned in the previous chapters of this masterclass! You also send your telemetry data to your New Relic account and see how useful the data is for monitoring your system and observing its behaviors.

If you haven’t already, sign up for a free New Relic account. You need one to complete this workshop.

Set up your environment

Step 1 of 2

Clone our demo repository:

bash

$git clone https://github.com/newrelic-experimental/mcv3-apps/

Step 2 of 2

Change to the Uninstrumented/python directory:

bash

$cd mcv3-apps/Uninstrumented/python

Next, you familiarize yourself with the app logic.

Familiarize yourself with the application

This demo service uses Flask, a Python micro framework for building web applications.

In Uninstrumented/python, you'll find a Python module called main.py. This holds the logic for your service. It has a single endpoint, called /fibonacci, that takes an argument, n, calculates the nth fibonacci number, and returns the results in a JSON-serialized format:

from flask import Flask, jsonify, request

app = Flask(__name__)

@app.errorhandler(ValueError)
def handle_value_exception(error):
    response = jsonify(message=str(error))
    response.status_code = 400
    return response

@app.route("/fibonacci")
def fib():
    n = request.args.get("n", None)
    return jsonify(n=n, result=calcfib(n))

def calcfib(n):
    try:
        n = int(n)
        assert 1 <= n <= 90
    except (ValueError, AssertionError) as e:
        raise ValueError("n must be between 1 and 90") from e

    b, a = 0, 1  # b, a initialized as F(0), F(1)
    for _ in range(1, n):
        b, a = a, a + b  # b, a always store F(i-1), F(i)
    return a

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=5000, debug=True)

main.py

The logic for calculating the nth fibonacci number is contained within a function called calcfib():

from flask import Flask, jsonify, request

app = Flask(__name__)

@app.errorhandler(ValueError)
def handle_value_exception(error):
    response = jsonify(message=str(error))
    response.status_code = 400
    return response

@app.route("/fibonacci")
def fib():
    n = request.args.get("n", None)
    return jsonify(n=n, result=calcfib(n))

def calcfib(n):
    try:
        n = int(n)
        assert 1 <= n <= 90
    except (ValueError, AssertionError) as e:
        raise ValueError("n must be between 1 and 90") from e

    b, a = 0, 1  # b, a initialized as F(0), F(1)
    for _ in range(1, n):
        b, a = a, a + b  # b, a always store F(i-1), F(i)
    return a

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=5000, debug=True)

main.py

This function takes the argument, n, and converts it to an integer. Then, it checks if it’s between 1 and 90. If it’s out of bounds or it’s not an integer, the function raises a ValueError and rejects the request. Otherwise, it computes and returns the nth fibonacci number.

Those are the most important functions you need to know about this Python application before you instrument it. Next, you install your OpenTelemetry dependencies.

Install dependencies

Before you can instrument your application, you need to add some OpenTelemetry dependencies to your project. In the python directory, you’ll find a requirements.txt file. This holds the dependency requirements for the demo application.

Step 1 of 3

First, install the project’s existing dependencies into a virtual environment:

bash

$python3 -m venv venv
$source venv/bin/activate
$pip install -r requirements.txt

Step 2 of 3

Next, install the OpenTelemetry dependencies you need to instrument your application:

bash

$# Install the API
$pip install opentelemetry-api
$# Install the OTLP exporter
$pip install opentelemetry-exporter-otlp==1.11.0
$# Install the Flask auto-instrumentation
$pip install opentelemetry-instrumentation-flask==0.30b1

This not only installs these packages, but also their dependencies.

Step 3 of 3

Finally, store all your new package references in requirements.txt:

bash

$pip freeze > requirements.txt

This will let Docker install them when it builds your app container.

Now, you’re ready to instrument your app.

Instrument your application

You’ve set up your environment, read the code, and spun up your app and a load generator. Now, it’s time to instrument your app with OpenTelemetry!

Step 1 of 9

The first thing you need to do to instrument your application is to configure your SDK. There are a few components to this:

Create a resource that captures information about your app and telemetry environments.
Create a tracer provider that you configure with your resource.
Configure the trace API with your tracer provider.

These steps set up your API to know about its environment. The resource you created will be attached to spans that you create with the trace API.

In main.py, configure your SDK:

1
from flask import Flask, jsonify, request
2
from opentelemetry import trace
3
from opentelemetry.sdk.resources import Resource
4
from opentelemetry.sdk.trace import TracerProvider
5

6
trace.set_tracer_provider(
7
   TracerProvider(
8
       resource=Resource.create(
9
           {
10
               "service.name": "fibonacci",
11
               "service.instance.id": "2193801",
12
               "telemetry.sdk.name": "opentelemetry",
13
               "telemetry.sdk.language": "python",
14
               "telemetry.sdk.version": "0.13.dev0",
15
           }
16
       ),
17
   ),
18
)
19

20
app = Flask(__name__)
21

22
@app.errorhandler(ValueError)
23
def handle_value_exception(error):
24
    response = jsonify(message=str(error))
25
    response.status_code = 400
26
    return response
27

28
@app.route("/fibonacci")
29
def fib():
30
    n = request.args.get("n", None)
31
    return jsonify(n=n, result=calcfib(n))
32

33
def calcfib(n):
34
    try:
35
        n = int(n)
36
        assert 1 <= n <= 90
37
    except (ValueError, AssertionError) as e:
38
        raise ValueError("n must be between 1 and 90") from e
39

40
    b, a = 0, 1  # b, a initialized as F(0), F(1)
41
    for _ in range(1, n):
42
        b, a = a, a + b  # b, a always store F(i-1), F(i)
43
    return a
44

45
if __name__ == "__main__":
46
    app.run(host="0.0.0.0", port=5000, debug=True)

main.py

1
version: '3'
2
services:
3
  fibonacci:
4
    build: ./
5
    ports:
6
      - "8080:5000"
7
  load-generator:
8
    build: ./load-generator

docker-compose.yaml

Here, you imported the trace API from the opentelemetry package and the TracerProvider and Resource classes from the SDK.

Then, you created a new tracer provider. It references a resource you use to describe your environment. Notice that these resource attributes adhere to the semantic conventions you learned about in the previous chapters.

Finally, you supply the tracer provider to the trace API.

Step 2 of 9

Next, you need to configure how you want to process and export spans in your application.

Add a span processor, and attach it to your tracer provider:

1
from flask import Flask, jsonify, request
2
from grpc import Compression
3
from opentelemetry import trace
4
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
5
from opentelemetry.sdk.resources import Resource
6
from opentelemetry.sdk.trace import TracerProvider
7
from opentelemetry.sdk.trace.export import BatchSpanProcessor
8

9
trace.set_tracer_provider(
10
   TracerProvider(
11
       resource=Resource.create(
12
           {
13
               "service.name": "fibonacci",
14
               "service.instance.id": "2193801",
15
               "telemetry.sdk.name": "opentelemetry",
16
               "telemetry.sdk.language": "python",
17
               "telemetry.sdk.version": "0.13.dev0",
18
           }
19
       ),
20
   ),
21
)
22

23
trace.get_tracer_provider().add_span_processor(
24
   BatchSpanProcessor(OTLPSpanExporter(compression=Compression.Gzip))
25
)
26

27
app = Flask(__name__)
28

29
@app.errorhandler(ValueError)
30
def handle_value_exception(error):
31
    response = jsonify(message=str(error))
32
    response.status_code = 400
33
    return response
34

35
@app.route("/fibonacci")
36
def fib():
37
    n = request.args.get("n", None)
38
    return jsonify(n=n, result=calcfib(n))
39

40
def calcfib(n):
41
    try:
42
        n = int(n)
43
        assert 1 <= n <= 90
44
    except (ValueError, AssertionError) as e:
45
        raise ValueError("n must be between 1 and 90") from e
46

47
    b, a = 0, 1  # b, a initialized as F(0), F(1)
48
    for _ in range(1, n):
49
        b, a = a, a + b  # b, a always store F(i-1), F(i)
50
    return a
51

52
if __name__ == "__main__":
53
    app.run(host="0.0.0.0", port=5000, debug=True)

main.py

1
version: '3'
2
services:
3
  fibonacci:
4
    build: ./
5
    ports:
6
      - "8080:5000"
7
  load-generator:
8
    build: ./load-generator

docker-compose.yaml

Here, you use a BatchSpanProcessor, which groups spans as they finish before sending them to the span exporter. The span exporter you use is the built-in OTLPSpanExporter with gzip compression. This allows you to efficiently send your span data to any native OTLP endpoint.

Step 3 of 9

The exporter you configured in the last step needs two important values:

A location where you want to send the data (New Relic, in this case)
An API key so that New Relic can accept your data and associate it to your account

In Uninstrumented/python/docker-compose.yaml, configure your OTLP exporter:

1
from flask import Flask, jsonify, request
2
from grpc import Compression
3
from opentelemetry import trace
4
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
5
from opentelemetry.sdk.resources import Resource
6
from opentelemetry.sdk.trace import TracerProvider
7
from opentelemetry.sdk.trace.export import BatchSpanProcessor
8

9
trace.set_tracer_provider(
10
   TracerProvider(
11
       resource=Resource.create(
12
           {
13
               "service.name": "fibonacci",
14
               "service.instance.id": "2193801",
15
               "telemetry.sdk.name": "opentelemetry",
16
               "telemetry.sdk.language": "python",
17
               "telemetry.sdk.version": "0.13.dev0",
18
           }
19
       ),
20
   ),
21
)
22

23
trace.get_tracer_provider().add_span_processor(
24
   BatchSpanProcessor(OTLPSpanExporter(compression=Compression.Gzip))
25
)
26

27
app = Flask(__name__)
28

29
@app.errorhandler(ValueError)
30
def handle_value_exception(error):
31
    response = jsonify(message=str(error))
32
    response.status_code = 400
33
    return response
34

35
@app.route("/fibonacci")
36
def fib():
37
    n = request.args.get("n", None)
38
    return jsonify(n=n, result=calcfib(n))
39

40
def calcfib(n):
41
    try:
42
        n = int(n)
43
        assert 1 <= n <= 90
44
    except (ValueError, AssertionError) as e:
45
        raise ValueError("n must be between 1 and 90") from e
46

47
    b, a = 0, 1  # b, a initialized as F(0), F(1)
48
    for _ in range(1, n):
49
        b, a = a, a + b  # b, a always store F(i-1), F(i)
50
    return a
51

52
if __name__ == "__main__":
53
    app.run(host="0.0.0.0", port=5000, debug=True)

main.py

1
version: '3'
2
services:
3
  fibonacci:
4
    build: ./
5
    ports:
6
      - "8080:5000"
7
    environment:
8
     OTEL_EXPORTER_OTLP_ENDPOINT: https://otlp.nr-data.net:4317
9
     OTEL_EXPORTER_OTLP_HEADERS: "api-key=${NEW_RELIC_API_KEY}"
10
  load-generator:
11
    build: ./load-generator

docker-compose.yaml

The first variable you set configures the exporter to send telemetry data to https://otlp.nr-data.net:4317. This is our US-based OTLP endpoint. If you’re in the EU, use https://otlp.eu01.nr-data.net instead.

The second variable passes the NEW_RELIC_API_KEY from your local machine in the api-key header of your OTLP requests. You set this environment variable in the next step.

The OTLP exporter looks for these variables in the Docker container’s environment and applies them at runtime.

Step 4 of 9

Get or create a New Relic license key for your account, and set it in an environment variable called NEW_RELIC_API_KEY:

bash

$export NEW_RELIC_API_KEY=<YOUR_LICENSE_KEY>

Important

Don’t forget to replace <YOUR_LICENSE_KEY> with your real license key!

Step 5 of 9

Back in main.py, instrument your Flask application:

1
from flask import Flask, jsonify, request
2
from grpc import Compression
3
from opentelemetry import trace
4
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
5
from opentelemetry.instrumentation.flask import FlaskInstrumentor
6
from opentelemetry.sdk.resources import Resource
7
from opentelemetry.sdk.trace import TracerProvider
8
from opentelemetry.sdk.trace.export import BatchSpanProcessor
9

10
trace.set_tracer_provider(
11
   TracerProvider(
12
       resource=Resource.create(
13
           {
14
               "service.name": "fibonacci",
15
               "service.instance.id": "2193801",
16
               "telemetry.sdk.name": "opentelemetry",
17
               "telemetry.sdk.language": "python",
18
               "telemetry.sdk.version": "0.13.dev0",
19
           }
20
       ),
21
   ),
22
)
23

24
trace.get_tracer_provider().add_span_processor(
25
   BatchSpanProcessor(OTLPSpanExporter(compression=Compression.Gzip))
26
)
27

28
app = Flask(__name__)
29
FlaskInstrumentor().instrument_app(app)
30

31
@app.errorhandler(ValueError)
32
def handle_value_exception(error):
33
    response = jsonify(message=str(error))
34
    response.status_code = 400
35
    return response
36

37
@app.route("/fibonacci")
38
def fib():
39
    n = request.args.get("n", None)
40
    return jsonify(n=n, result=calcfib(n))
41

42
def calcfib(n):
43
    try:
44
        n = int(n)
45
        assert 1 <= n <= 90
46
    except (ValueError, AssertionError) as e:
47
        raise ValueError("n must be between 1 and 90") from e
48

49
    b, a = 0, 1  # b, a initialized as F(0), F(1)
50
    for _ in range(1, n):
51
        b, a = a, a + b  # b, a always store F(i-1), F(i)
52
    return a
53

54
if __name__ == "__main__":
55
    app.run(host="0.0.0.0", port=5000, debug=True)

main.py

1
version: '3'
2
services:
3
  fibonacci:
4
    build: ./
5
    ports:
6
      - "8080:5000"
7
    environment:
8
     OTEL_EXPORTER_OTLP_ENDPOINT: https://otlp.nr-data.net:4317
9
     OTEL_EXPORTER_OTLP_HEADERS: "api-key=${NEW_RELIC_API_KEY}"
10
  load-generator:
11
    build: ./load-generator

docker-compose.yaml

Here, you import and use the FlaskInstrumentor. This provides automatic base instrumentation for Flask applications.

Step 6 of 9

Now that you've automatically instrumented your Flask application, you'll instrument calcfib(). This will let you create a span that's specifically scoped to the function where you can add custom attributes and events.

In the last chapter, you learned that you need a tracer to create spans, so get a tracer from the trace API:

1
from flask import Flask, jsonify, request
2
from grpc import Compression
3
from opentelemetry import trace
4
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
5
from opentelemetry.instrumentation.flask import FlaskInstrumentor
6
from opentelemetry.sdk.resources import Resource
7
from opentelemetry.sdk.trace import TracerProvider
8
from opentelemetry.sdk.trace.export import BatchSpanProcessor
9

10
trace.set_tracer_provider(
11
   TracerProvider(
12
       resource=Resource.create(
13
           {
14
               "service.name": "fibonacci",
15
               "service.instance.id": "2193801",
16
               "telemetry.sdk.name": "opentelemetry",
17
               "telemetry.sdk.language": "python",
18
               "telemetry.sdk.version": "0.13.dev0",
19
           }
20
       ),
21
   ),
22
)
23

24
trace.get_tracer_provider().add_span_processor(
25
   BatchSpanProcessor(OTLPSpanExporter(compression=Compression.Gzip))
26
)
27

28
app = Flask(__name__)
29
FlaskInstrumentor().instrument_app(app)
30

31
tracer = trace.get_tracer(__name__)
32

33
@app.errorhandler(ValueError)
34
def handle_value_exception(error):
35
    response = jsonify(message=str(error))
36
    response.status_code = 400
37
    return response
38

39
@app.route("/fibonacci")
40
def fib():
41
    n = request.args.get("n", None)
42
    return jsonify(n=n, result=calcfib(n))
43

44
def calcfib(n):
45
    try:
46
        n = int(n)
47
        assert 1 <= n <= 90
48
    except (ValueError, AssertionError) as e:
49
        raise ValueError("n must be between 1 and 90") from e
50

51
    b, a = 0, 1  # b, a initialized as F(0), F(1)
52
    for _ in range(1, n):
53
        b, a = a, a + b  # b, a always store F(i-1), F(i)
54
    return a
55

56
if __name__ == "__main__":
57
    app.run(host="0.0.0.0", port=5000, debug=True)

main.py

1
version: '3'
2
services:
3
  fibonacci:
4
    build: ./
5
    ports:
6
      - "8080:5000"
7
    environment:
8
     OTEL_EXPORTER_OTLP_ENDPOINT: https://otlp.nr-data.net:4317
9
     OTEL_EXPORTER_OTLP_HEADERS: "api-key=${NEW_RELIC_API_KEY}"
10
  load-generator:
11
    build: ./load-generator

docker-compose.yaml

trace.get_tracer() is a convenience function that returns a tracer from the tracer provider you configured in a previous step.

Step 7 of 9

Add some custom instrumentation to calcfib():

1
from flask import Flask, jsonify, request
2
from grpc import Compression
3
from opentelemetry import trace
4
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
5
from opentelemetry.instrumentation.flask import FlaskInstrumentor
6
from opentelemetry.sdk.resources import Resource
7
from opentelemetry.sdk.trace import TracerProvider
8
from opentelemetry.sdk.trace.export import BatchSpanProcessor
9

10
trace.set_tracer_provider(
11
   TracerProvider(
12
       resource=Resource.create(
13
           {
14
               "service.name": "fibonacci",
15
               "service.instance.id": "2193801",
16
               "telemetry.sdk.name": "opentelemetry",
17
               "telemetry.sdk.language": "python",
18
               "telemetry.sdk.version": "0.13.dev0",
19
           }
20
       ),
21
   ),
22
)
23

24
trace.get_tracer_provider().add_span_processor(
25
   BatchSpanProcessor(OTLPSpanExporter(compression=Compression.Gzip))
26
)
27

28
app = Flask(__name__)
29
FlaskInstrumentor().instrument_app(app)
30

31
tracer = trace.get_tracer(__name__)
32

33
@app.errorhandler(ValueError)
34
def handle_value_exception(error):
35
    response = jsonify(message=str(error))
36
    response.status_code = 400
37
    return response
38

39
@app.route("/fibonacci")
40
def fib():
41
    n = request.args.get("n", None)
42
    return jsonify(n=n, result=calcfib(n))
43

44
def calcfib(n):
45
    with tracer.start_as_current_span("fibonacci") as span:
46
        span.set_attribute("fibonacci.n", n)
47

48
        try:
49
            n = int(n)
50
            assert 1 <= n <= 90
51
        except (ValueError, AssertionError) as e:
52
            raise ValueError("n must be between 1 and 90") from e
53

54
        b, a = 0, 1  # b, a initialized as F(0), F(1)
55
        for _ in range(1, n):
56
            b, a = a, a + b  # b, a always store F(i-1), F(i)
57

58
        span.set_attribute("fibonacci.result", a)
59
        return a
60

61
if __name__ == "__main__":
62
    app.run(host="0.0.0.0", port=5000, debug=True)

main.py

1
version: '3'
2
services:
3
  fibonacci:
4
    build: ./
5
    ports:
6
      - "8080:5000"
7
    environment:
8
     OTEL_EXPORTER_OTLP_ENDPOINT: https://otlp.nr-data.net:4317
9
     OTEL_EXPORTER_OTLP_HEADERS: "api-key=${NEW_RELIC_API_KEY}"
10
  load-generator:
11
    build: ./load-generator

docker-compose.yaml

First, you started a new span, called "fibonacci", with the start_as_current_span() context manager. start_as_current_span() sets your new span as the current span, which captures any updates that happen within the context.

Then, you add an attribute, called "fibonacci.n", that holds the value of the requested number in the fibonacci sequence.

Finally, if the function successfully computes the nth fibonacci number, you set another attribute, called "fibonacci.result", with the result.

Step 8 of 9

By default, start_as_current_span() captures data from uncaught exceptions within the scope of the span. In your function, you raise such an exception:

raise ValueError("n must be between 1 and 90") from e

start_as_current_span() uses this exception to automatically add an exception span event to your span. But it doesn't do the same for the root span.

Set an error status on the root span of your trace:

1
from flask import Flask, jsonify, request
2
from grpc import Compression
3
from opentelemetry import trace
4
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
5
from opentelemetry.instrumentation.flask import FlaskInstrumentor
6
from opentelemetry.sdk.resources import Resource
7
from opentelemetry.sdk.trace import TracerProvider
8
from opentelemetry.sdk.trace.export import BatchSpanProcessor
9
from opentelemetry.trace.status import Status, StatusCode
10

11
trace.set_tracer_provider(
12
   TracerProvider(
13
       resource=Resource.create(
14
           {
15
               "service.name": "fibonacci",
16
               "service.instance.id": "2193801",
17
               "telemetry.sdk.name": "opentelemetry",
18
               "telemetry.sdk.language": "python",
19
               "telemetry.sdk.version": "0.13.dev0",
20
           }
21
       ),
22
   ),
23
)
24

25
trace.get_tracer_provider().add_span_processor(
26
   BatchSpanProcessor(OTLPSpanExporter(compression=Compression.Gzip))
27
)
28

29
app = Flask(__name__)
30
FlaskInstrumentor().instrument_app(app)
31

32
tracer = trace.get_tracer(__name__)
33

34
@app.errorhandler(ValueError)
35
def handle_value_exception(error):
36
    trace.get_current_span().set_status(
37
        Status(StatusCode.ERROR, "Number outside of accepted range.")
38
    )
39
    response = jsonify(message=str(error))
40
    response.status_code = 400
41
    return response
42

43
@app.route("/fibonacci")
44
def fib():
45
    n = request.args.get("n", None)
46
    return jsonify(n=n, result=calcfib(n))
47

48
def calcfib(n):
49
    with tracer.start_as_current_span("fibonacci") as span:
50
        span.set_attribute("fibonacci.n", n)
51

52
        try:
53
            n = int(n)
54
            assert 1 <= n <= 90
55
        except (ValueError, AssertionError) as e:
56
            raise ValueError("n must be between 1 and 90") from e
57

58
        b, a = 0, 1  # b, a initialized as F(0), F(1)
59
        for _ in range(1, n):
60
            b, a = a, a + b  # b, a always store F(i-1), F(i)
61

62
        span.set_attribute("fibonacci.result", a)
63
        return a
64

65
if __name__ == "__main__":
66
    app.run(host="0.0.0.0", port=5000, debug=True)

main.py

1
version: '3'
2
services:
3
  fibonacci:
4
    build: ./
5
    ports:
6
      - "8080:5000"
7
    environment:
8
     OTEL_EXPORTER_OTLP_ENDPOINT: https://otlp.nr-data.net:4317
9
     OTEL_EXPORTER_OTLP_HEADERS: "api-key=${NEW_RELIC_API_KEY}"
10
  load-generator:
11
    build: ./load-generator

docker-compose.yaml

Here, you manually set the root span's status to StatusCode.ERROR when you handle the ValueError. Having an error status on the root span will be useful in finding traces with errors in New Relic.

Step 9 of 9

In the same shell you used to export your environment variable, navigate to Uninstrumented/python, then spin up the project's containers with Docker Compose:

bash

$docker-compose up --build

This runs two docker services:

fibonacci: Your app service
load-generator: A service that simulates traffic to your app

The load generator makes periodic requests to your application. It sends a mixture of requests that you expect to succeed and ones that you expect to fail. Looking at the Docker Compose log stream, you should see that both your application and load generator are running.

You’re now ready to view your data in New Relic.

Course

This lesson is a part of our OpenTelemetry masterclass. Continue on to the next lesson: View a summary of your data.