[Feature Request]: Stress Testing

[sameersegal]

Is your feature request related to a problem? Please describe.

To build confidence in JB Manager architecture. We need to stress test the architecture.

Describe the solution you'd like

We need to mock the channel to generate the volume of 10 / 100 / 1000 messages a minute and see the delivery time of each message.

Scenarios:

1. Keep the FSM very simple -- echo the message back to the user -- no LLM
2. Using above FSM, send the message in voice format -- convert speech to text, translate from hindi to english but no LLM
3. Add an LLM call into the FSM but use GPT3.5-Turbo

Steps:

1. Make an API request to generate message. Generate a unique mobile number / user id as part of the payload.
2. Change the channel API endpoint and point it to a new server. Add a server that simply logs in coming message and returns 200.
3. Compare the delay between input and output based on the unique mobile number / user id.

Additional context

No response

[KaranrajM]

@wi0lono Do we have any updates on this?

[Lekhanrao]

@DevvStrange, any update on this please?

[Lekhanrao]

Assigned ticket to @ananyaagrawal23

[Lekhanrao]

@ananyaagrawal23, Wanted to check if you got a chance to look into this?

[ananyaagrawal23]

Hi @Lekhanrao, yes I am currently working on this issue.

[ananyaagrawal23]

Hi all,

Performed stress testing on my local machine (Macbook Pro) with following configurations:
Chip: M3 Pro
RAM: 36 GB
macOS version: Sonoma 14.7

Tested for all 3 given scenarios for 1, 10, 100 and 1000 user requests:

1. Sent text message and got average text response time.
2. Sent Voice message and got average audio response time
3. Added LLM call and got average response time for text response.

The report is attached here:
Stress Testing.xlsx

Also, the observations for each scenario are mentioned in the report itself.

[Lekhanrao]

@ananyaagrawal23 what are the next steps here?