Skip to main content

Quickstart: observe an existing consumer

If you already have a consumer running (any broker, any framework), you don't need to rewrite it to get visibility. Wrap each delivery in an observation and Observatory gets the outcome, the timing, and the trace lineage, and you get dashboard control back.

Before you start

You'll need an API key scoped to telemetry. Any admin can create one under Organization settings → API Keys. Keep it secret and treat it like a password.

  1. 1
    Install the SDK
    pip install "pymidil[auth]"
  2. 2
    Create a ConsumerObserver
    Give it your Observatory URL, your API key, a name for this consumer (this is what shows up, and what you'll control, in the dashboard), and a label for the broker it reads from:
    from pymidil.event.observability import ConsumerObserver

    observe = ConsumerObserver(
      observatory_url="https://api.midil.io",
      api_key="mo_your_api_key",
      consumer="orders-worker",
      broker="kafka",  # any broker works, it's just a label
    )
  3. 3
    Wrap each delivery
    Drop the observation around the part of your loop that handles one message. Everything inside runs exactly as it did before:
    async for record in your_existing_consumer:  # your existing loop
      async with observe(record.key, "OrderPlaced", headers=record.headers):
          await handle_order(record)  # your code, untouched
    The event type string ("OrderPlaced" above) is whatever name you want that kind of event to show up as in the dashboard.
  4. 4
    Honor dashboard control (optional, recommended)
    Check control state before pulling the next message, so pausing/throttling a consumer from the dashboard actually does something:
    if not (await observe.control.get()).state.should_pull:
      continue  # paused or draining from the dashboard
    See the [Consumer control](/sdks/python/guides/consumer-control) guide for throttling and draining too.
  5. 5
    Check the dashboard
    Open Overview. Your first observed event should appear within moments.

ProducerObserver is the emit-side counterpart, for wrapping a service that publishes events rather than consumes them. Same idea, wrapped around your send call instead of your handler.

What's next​