Simple query utility for Amazon Managed Prometheus

python
aws
prometheus
Author

Erik Lundevall-Zara

Published

December 12, 2024

Recently I had the task to set up an Amazon Managed Prometheus (AMP) workspace. AMP is AWS managed version of Prometheus monitoring solution, primarily the server side that stored the metric data. The idea was to connect that with an already existing deployment of Grafana, running in an EKS cluster to visualize these metrics.

Setting up the AMP workspace itself was simple, and installing something in EKS clusters that would write to this workspace properly was a bit more tricky, but doable.

Unfortunately, AMP does not provide any simple interface or API to query its data, and the CloudWatch logs reportedly would only log warnings or errors.

I could indirectly see through CloudWatch metrics that metrics data was ingested. But my attempts to connect it to Grafana did not work, using the plugin for AMP. One of the configuration settings for this plugin was the version of Prometheus, but I had no idea which version AMP uses. There is no documentation, and I did not find anyway to get that from AWS CLI.

So my approach was then to use the Prometheus query API and ask AMP what version of Prometheus it is using. The metric that Prometheus provides that contains that information is prometheus_build_info.

So I had to use this API. But the request itself must also be signed using AWS SigV4 authentication, and I need to have the right credentials to access the workspace.

Luckily there are ways to do that, and a suggestion from claude.ai pointed to the requests-aws4auth library.

With a little bit of work I set up a small utility in Python that would send a query to AMP and print out the result. I combined that with one of my favourite Python tools, the excellent uv. This resulted in a single Python script that I could run with uv run queryamp.py, with dependencies embedded in the script itself.

This helped me get the build info for AMP, and also help with some additional checks and testing of the data that had been collected.

❯ uv run queryamp.py
Reading inline script metadata from: queryamp.py
Status code: 200
Query: prometheus_build_info
Response: {"status":"success","data":{"resultType":"vector","result":[{"metric":{"__name__":"prometheus_build_info","branch":"HEAD","goarch":"amd64","goos":"linux","goversion":"go1.23.2","instance":"localhost:9090","job":"prometheus","revision":"6d7569113f1ca814f1e149f74176656540043b8d","tags":"netgo,builtinassets,stringlabels","version":"2.55.1"},"value":[1733995459,"1"]}]}}

The script itself if fairly simple, and I found it useful to run various queries for a quick check. I hope you find it useful.

# /// script
# requires-python = ">=3.13"
# dependencies = [
#     "boto3",
#     "requests",
#     "requests-aws4auth",
# ]
# ///
import os
import argparse
import requests
from requests_aws4auth import AWS4Auth
import boto3

region = os.getenv("AWS_REGION", os.getenv("AWS_DEFAULT_REGION", "eu-north-1"))
workspace_id = os.getenv("WORKSPACE_ID")
query = "prometheus_build_info"

parser = argparse.ArgumentParser(prog="queryamp", description="Query AMP workspace")
parser.add_argument(
    "-w",
    "--workspace_id",
    type=str,
    help="AMP Workspace ID",
    nargs="?",
    default=workspace_id,
)
parser.add_argument(
    "-r", "--region", type=str, help="AWS Region", nargs="?", default=region
)
parser.add_argument("query", type=str, help="PromQL Query", nargs="?", default=query)

args = parser.parse_args()
region = args.region
workspace_id = args.workspace_id
query = args.query

session = boto3.Session(region_name=region)
credentials = session.get_credentials()

auth = AWS4Auth(
    credentials.access_key,
    credentials.secret_key,
    region,
    "aps",
    session_token=credentials.token if credentials.token else None,
)

url = f"https://aps-workspaces.{region}.amazonaws.com/workspaces/{workspace_id}/api/v1/query"

response = requests.get(url, params={"query": query}, auth=auth)

print(f"Status code: {response.status_code}")
print(f"Query: {query}")
print(f"Response: {response.text}")
Back to top