Enhancing Message Extensibility and Serializer Customization

[llupa]

Context:
The recent integration of the Symfony Serializer has standardized object normalization within the library. However, the current design presents challenges for developers aiming to extend message functionalities with business-specific metadata, such as timestamps for session continuity.

Identified Challenges:

1. Final Message Class:
   The *Message class is declared as final, preventing inheritance and thus limiting the ability to introduce custom properties or behaviors tailored to specific business needs.

final readonly class SystemMessage implements MessageInterface

Similar final declarations exist for UserMessage, AssistantMessage, and ToolCallMessage. Because these classes cannot be extended, it is difficult to attach additional state (timestamp example) while still leveraging the library’s serializer.

2. Serializer Accessibility:
   The serializer instance is currently private and inlined within the Platform class, restricting the injection of custom normalizers or encoders necessary for handling extended message structures.

public function __construct(
    iterable $modelClients,
    iterable $responseConverter,
    private ?Contract $contract = null,
) {
    $this->contract = $contract ?? Contract::create();

public static function create(NormalizerInterface ...$normalizer): self
{
    // Messages
    $normalizer[] = new MessageBagNormalizer();
    $normalizer[] = new AssistantMessageNormalizer();
    ...
    return new self(
        new Serializer($normalizer),
    );
}

3. Rigid Normalizer Configuration:

   Since there is no public API to inject additional normalizers after a Platform is constructed, customizing serialization for extended message classes requires rebuilding the entire serializer.

   Adding custom normalizers requires modifications across multiple platform factories or constructors, increasing the complexity and maintenance overhead for developers seeking to customize serialization behavior.

PhpLlm\LlmChain\Platform\Platform.php
PhpLlm\LlmChain\Platform\Bridge\Anthropic\PlatformFactory.php
PhpLlm\LlmChain\Platform\Bridge\Azure\OpenAI\PlatformFactory.php
PhpLlm\LlmChain\Platform\Bridge\Bedrock\Platform.php // sidenote: no factory?
PhpLlm\LlmChain\Platform\Bridge\Google\PlatformFactory.php
PhpLlm\LlmChain\Platform\Bridge\HuggingFace\PlatformFactory.php
PhpLlm\LlmChain\Platform\Bridge\Mistral\PlatformFactory.php
PhpLlm\LlmChain\Platform\Bridge\OpenAI\PlatformFactory.php
PhpLlm\LlmChain\Platform\Bridge\OpenRouter\PlatformFactory.php
PhpLlm\LlmChain\Platform\Bridge\Replicate\PlatformFactory.php

Proposed Enhancements:

1. Make the Message Classes Extensible:
   Remove the final keyword from the *Message specific classes to allow developers to subclass and introduce additional properties, such as timestamps or other business specific context data, facilitating richer context management.

2. Expose Platform Serializer for decoration:
   Expose the constructed serializers so that user-land code can decorate and use their own extra normalizers if needed.

3. Optional: Streamline Normalizer Integration:
   Introduce a more flexible system for registering custom normalizers, possibly through service tagging or configuration options, to simplify the process of extending serialization logic across different platforms.

Benefits:

• Empowers developers to tailor message structures to their specific business requirements.
• Enhances the modularity and maintainability of the codebase by promoting cleaner extension points.

[OskarStark]

PhpLlm\LlmChain\Platform\Bridge\Bedrock\Platform.php // sidenote: no factory?

should be renamed I guess @chr-hertel

[OskarStark]

If I understand correctly, the current normalizers are designed for internal use only and not intended for userland consumption. However, I could be mistaken about this.

[llupa]

@OskarStark Yes indeed this is what I understand from the code. But adding more business context is severely gutted. You have to either rely on the messages that the library offers or, fork the repo and hack it to your liking.

I still think that decorating the serializer objects can keep both the lib happy and user-land can add more fields that they want to enhance their use case.

Recently a MessageStoreInteface was introduced and this can be very powerful for chat-bots type of projects because you can "pick up" a conversation where you left it, and say you add things like a timestamp to your messages, then the model could have a sense of how long has it been since we spoke.

Alternatively, messages could also have user land identifiers too, like product identifier, so that the conversation is put in the context of a particular product of Acme Online Store and so on.

---

There is a chance that maybe I am not getting how these little extra contexts are supposed to work, because they could also be tool calls, but, if you need to ensure certain data are part of the session, and not a tool that might be or not called by the model, then adding an option to extend is the way to go in my opinion.

[OskarStark]

I agree with you that this would be a nice addition. Having access to normalizers for userland would enable more flexibility in customizing message serialization and handling platform-specific requirements.

[llupa]

@DZunke shouldn't it be:

// PhpLlm\LlmChain\Platform\Bridge\Anthropic\PlatformFactory

public static function create(
    #[\SensitiveParameter]
    string $apiKey,
    string $version = '2023-06-01',
    ?HttpClientInterface $httpClient = null,
    iterable $extraNormalizers = [], // This would be new
): Platform {
    $httpClient = $httpClient instanceof EventSourceHttpClient ? $httpClient : new EventSourceHttpClient($httpClient);

    return new Platform(
        [new ModelClient($httpClient, $apiKey, $version)],
        [new ResponseConverter()],
        $contract ?? AnthropicContract::create($extraNormalizers), // Build serializer with additional normalizers
    );
}

Edit: Wait, where did your comment go?

[DZunke]

@DZunke shouldn't it be:
Edit: Wait, where did your comment go?

I deleted it because it was nonsense, sorry. I am currently fiddeling a bit around in the code to maybe find a solution that looks clean. But it goes in the direction, yes ... with the AnthropicContract. So a combination of your initial code and my thoughts.

[llupa]

What do you think about making the Message specific classes extendable? From my PoV this could be a potential way of using it with Symfony #[Context]

use Symfony\Component\Serializer\Attribute\Context;
use Symfony\Component\Serializer\Normalizer\DateTimeNormalizer;

final readonly class MyUserMessage extends UserMessage
{
    public function __construct(
        #[Context([DateTimeNormalizer::FORMAT_KEY => \DateTimeInteface::RFC_3339])]
        public DateTimeImmutable $timestamp = new DateTimeImmutable(),
        ContentInterface ...$content,
    ) {
        parent::__construct(...$content);
    }
}

Also, for custom normalizers, from my PoV again, if the $serializer of a platform is decorate-able you can have something like this:

#[AsDecorator(decorates: 'llm_chain.platform.anthropic.serializer')]
final readonly class MyMessageNormalizer implements NormalizerInterface
{
    public function __construct(
        #[AutowireDecorated]
        private NormalizerInterface $decorated,
    ) {
    }

    public function supportsNormalization(mixed $data, ?string $format = null, array $context = []): bool
    {
        return $data instanceof MyMessage;
    }

    public function getSupportedTypes(?string $format): array
    {
        return [
            MyMessage::class,
        ];
    }

    /**
     * @param MyMessage $data
     */
    public function normalize(mixed $data, ?string $format = null, array $context = []): array
    {
        $normalizedData = $this->decorated->normalize($data, $format, $context);

        if ($normalizedData['timestamp'] < (new DateTimeImmutable('1 month ago')->format(\DateTimeInterface::RFC_3339)) {
            $normalizedData['returning_user'] = true;
        }

        return $normalizedData;
    }

I know that the specifics might differ, but this is the general idea I had from my use case.

[DZunke]

I, for myself, have no problem with the messages being final. For this there is the MessageInterface to create custom messages, if needed. That is why i support the flexible configuration of the contract. So i would fix this side to bring flexibility together, again.

I have thrown a bunch of code in the ring with #372 - maybe this can change at least one side of the medal for you?

[llupa]

I have to try out the branch, just by looking at it I think having custom messages per platform will require a lot of plumbing on my side 😅

BTW Thanks a lot for all your work!

[chr-hertel]

Thanks @llupa for raising this and @DZunke for the first solution proposal.

I think introducing the sets (or specific bridge contract subclasses) is a good step and makes the factory code cleaner anyways.

I'm currently not a big fan of having the MessageInterface or MessageBagInterface - and would argue against them currently. buut, rather an idea forward: what about introducing the JsonSerializableNormalizer as part of the default contract so we could still just have simple custom implementations that would implement the JsonSerializable interface.

haven't tried it yet, but could be a good option - and i feel like i went too far with those normalizers all over the place anyways :D

[DZunke]

For me, adding the default JsonSerializable support back again would also be a good solution. It would also reintroduce the possibility to have custom objects, based on the interfaces, that are usable by the library. Having the normalizer layer internally feels not wrong in general but brought in a small async between flexible interfaces and default behaviour.

But in general having the possibility to change the contract on purpose feels not wrong to allow more flexibility aspects.