Documentation Index
Fetch the complete documentation index at: https://mintlify.com/Arize-ai/openinference/llms.txt
Use this file to discover all available pages before exploring further.
Python auto-instrumentation library for DSPy.
These traces are fully OpenTelemetry-compatible and can be sent to an OpenTelemetry collector for viewing, such as Arize Phoenix.
Installation
pip install openinference-instrumentation-dspy
Quickstart
This quickstart shows you how to instrument your DSPy application. It is adapted from the DSPy quickstart.
Install required packages:
pip install openinference-instrumentation-dspy dspy-ai arize-phoenix opentelemetry-sdk opentelemetry-exporter-otlp
Start Phoenix server
Start Phoenix in the background as a collector. By default, it listens on http://localhost:6006:
python -m phoenix.server.main serve
Setup instrumentation
Set up DSPyInstrumentor to trace your DSPy application and send traces to Phoenix:
from openinference.instrumentation.dspy import DSPyInstrumentor
from opentelemetry import trace as trace_api
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk import trace as trace_sdk
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
endpoint = "http://127.0.0.1:6006/v1/traces"
tracer_provider = trace_sdk.TracerProvider()
trace_api.set_tracer_provider(tracer_provider)
tracer_provider.add_span_processor(SimpleSpanProcessor(OTLPSpanExporter(endpoint)))
DSPyInstrumentor().instrument()
import dspy
from dspy.datasets.gsm8k import GSM8K, gsm8k_metric
turbo = dspy.OpenAI(model='gpt-3.5-turbo-instruct', max_tokens=250)
dspy.settings.configure(lm=turbo)
gms8k = GSM8K()
gsm8k_trainset, gsm8k_devset = gms8k.train[:10], gms8k.dev[:10]
Define a custom program
Define a program that utilizes the ChainOfThought module to perform step-by-step reasoning:
class CoT(dspy.Module):
def __init__(self):
super().__init__()
self.prog = dspy.ChainOfThought("question -> answer")
def forward(self, question):
return self.prog(question=question)
Optimize your program
Optimize your program using the BootstrapFewShot teleprompter:
from dspy.teleprompt import BootstrapFewShot
config = dict(max_bootstrapped_demos=4, max_labeled_demos=4)
teleprompter = BootstrapFewShot(metric=gsm8k_metric, **config)
optimized_cot = teleprompter.compile(CoT(), trainset=gsm8k_trainset, valset=gsm8k_devset)
from dspy.evaluate import Evaluate
evaluate = Evaluate(devset=gsm8k_devset, metric=gsm8k_metric, num_threads=4, display_progress=True, display_table=0)
evaluate(optimized_cot)
http://localhost:6006 to see your traces.
More Info