Pluggable Resources

In a multi-agent system, agents will have vastly different needs. This may be as simple as choosing a different system prompt, or as complex as using a different LLM model. Eidolon allows you to specify this customization without ever needing to jump into code. What’s more, it also allows you to override deeply nested configuration as well as define custom resources to use as templates.

Why

LLM applications require a ton of tweaking 🔨. Jumping into code for each and every code means this fiddling is slower and more error-prone. That is why at Eidolon 👻 we separated prompting and system configuration into simple kubernetes-like yaml files that can be modified without needing to open up a code editor 🔡.

Beyond simply speeding up cycle time 🚀, this also allows more personas 🧑‍🚒🧑‍🏫🧑‍🎨 to work on the application without needing deep understanding of the codebase 🔡. This allows different personas to focus on architecture, fine-tuning, model selection, and prompt engineering so each person can focus on their domain expertise without getting bogged down.

How

Spec Customization

In Eidolon’s agent yaml files, you can have probably noticed the spec field. This is where you can customize portions of the component’s configuration.

apiVersion: eidolon/v1
kind: Agent
metadata:
  name: qa
spec: ...

Field Customization

You can override individual fields of the component.

spec:
  title_generation_mode: "none"   # 🔎 overrides the component's default title_generation_mode

Implementation Customization

Most components have a default implementation that they point to. Agents use the SimpleAgent template by default. You can override this implementation, but remember the new contents need to match the new implementation’s spec pattern.

spec:
  implementation: RetrieverAgent  # 🔎 defines the new component to use
  loader_pattern: "**/*.test"     # 🔎 overrides the component's default loader_pattern
#  title_generation_mode: "none"  # 🚨 removed since it is now invalid

Nested Spec Customization

Components can also make References to other components. These subcomponents can be customized as well.

spec:
  apu:                            # 🔎 modify the component's default apu
    allow_tool_errors: false      # 🔎 by overriding a portion of the apu spec

Nested Implementation Customization

Of course you can also override nested implementations.

spec:
  apu:
    implementation: ConversationalAPU

This implementation pointer override is quite common, so we have added as shortcut for it. If you just specify a string for the reference, we will treat it as an implementation override.

spec:
  apu: ConversationalAPU

Reusable References

Now component customization is nice for experimentation, but it can get quite verbose if we need to constantly make the same customization to multiple components. For this reason, we have added References. These are reusable configurations that can be referenced by multiple components.

Let’s create a named resource frugal_apu so we can easily point to an older model for some agents.

frugal_apu.yaml

apiVersion: eidolon/v1
kind: Reference
metadata:
  name: frugal_apu

spec:
  implementation: 'ConversationalAPU'
  llm_unit:
    force_json: 'True'
    max_tokens: '3000'
    model: gpt-3.5-turbo-1106
    temperature: .1
  max_num_function_calls: 20

Now within our agent’s spec we can just reference this reference by name like we would any other component.

spec:
  apu: frugal_apu

We can continue to override sub components as you would expect.

spec:
  apu:
    implementation: frugal_apu
    max_num_function_calls: 10

Changing Default Components

Now what if you are a lean startup, and want to use the frugal_apu for all of your agents unless specified otherwise? You would prefer not to override the apu of every agent, you just want to set frugal_apu as the default apu.

apiVersion: eidolon/v1
kind: Reference
metadata:
  name: APU

spec: frugal_apu

This reference resource will change any component that references the APU to use our new frugal_apu instead of the default (ConversationalAPU).

Recipes

Most recipes will have some form of customization. Here are a few examples that use it more heavily.

GitHub Repo Expert

The RepoSearch overrides its loader with a different implementation to be able to read from github. On top of this it then customizes the files this loader can read.