Flow profiles by johnbilt · Pull Request #130 · bbc/tams

johnbilt · 2025-05-20T10:50:55Z

Details

Add the content of flow profiles to aid with the creation of flows in stores where there are house formats being used. Also significantly simplifies matching flows across edit by reference workflows

Jira Issue (if relevant)

Jira URL:

Related PRs

If merged before #115, add the new endpoints to the auth logic listings in that PR. If merged after, add that logic in this PR

Submitter PR Checks

(tick as appropriate)

PR completes task/fixes bug
API version has been incremented if necessary
ADR status has been updated, and ADR implementation has been recorded
Documentation updated (README, etc.)
PR added to Jira Issue (if relevant)
Follow-up stories added to Jira

Reviewer PR Checks

(tick as appropriate)

PR completes task/fixes bug
Design makes sense, and fits with our current code base
Code is easy to follow
PR size is sensible
Commit history is sensible and tidy

Info on PRs

The checks above are guidelines. They don't all have to be ticked, but they should all have been considered.

himslm01

A few readability improvements.

Updated wording Co-authored-by: James Sandford <james.sandford@bbc.co.uk>

… flow-profiles

samdbmg

Looks good to me overall: a good approach of maintaining flexibility while constraining in a way to make it easier to use.

I've inlined a handful of suggestions, and a couple of nits

samdbmg · 2025-07-31T15:31:44Z

I've also kicked off the CI for you, so you should be able to see if it passes validation

GeorginaShippey · 2025-08-05T17:12:10Z

+{
+  "type": "object",
+  "description": "Describes the technical characteristics of a Flow (common properties to all Flows, imported by type-specific specifications)",
+  "title": "Flow Core",


NIT: Would be good to have the title updated as it keeps the rendered version more readable, there are currently two schemas rendered out as Flow Core

Co-authored-by: Sam Mesterton-Gibbons <sam@samn.co.uk>

… flow-profiles

Co-authored-by: Georgina Shippey <9370167+GeorginaShippey@users.noreply.github.com>

… flow-profiles

j616 · 2026-05-13T16:21:22Z

+For workflows where replication of the same content formats are happening on a regular basis then it is recommended that the same Profile is loaded into both stores using the same UUID.
+This will mean than when Flows are replicated between the stores then the Profile identifier will continue to link to the metadata.
+
+If the Profile does not exist within the destination store then the Profile ID should continue to be preserved.


Do we need to describe how this interacts with the Profiles endpoint? Should the Profile be created if it doesn't exist? If not, how does it interact with the creation of a Profile with a matching ID? I think the paragraph below covers some of this. It might help if that is moved above this para?

I think there is a larger issue here which I'd not spotted. To create a flow you provide either a profile or the full flow metadata. If the profile does not exist in the store then you can't actually create the flow as there is no where to get the technical metadata from. So I think you either have to fail the creation and then the calling system either needs to create the profile and then re-try and flow creation or drop the profile and just create the flow directly. The first should be the preferred option as it retains the profile information.

j616 · 2026-05-13T16:33:41Z

+        It is up to the store implementation whether this is normalised on the flow creation or read-only.
+
+        When creating a flow using a Profile it is not possible to override any specific fields as this would invalidate the link to the Profile.  
+        Supplying a Profile ID and technical metadata should result in a 400 validation error


Suggested change

Supplying a Profile ID and technical metadata should result in a 400 validation error

Supplying a Profile ID and technical metadata defined within the specified Profile should result in a 400 validation error

The line above clearly says it is not possible to override the technical parameters of a profile, so the thinking was you supply a profile or technical metadata not both. So possibly might be better saying Supplying a Profile ID and any technical metadata should result in a 400 validation error

Sorry, yes. I had it in my head that a partial profile could be used. But I think that's a future iteration. I think your tweaked message helps. Have we explicitly defined what constitutes "technical metadata" anywhere? Specifically in the human readable version of the spec.

j616 · 2026-05-13T16:35:58Z

        "204":
          description: No content. The Flow has been updated.
        "400":
          description: Bad request. Invalid Flow JSON.


Do we need to add conflicting Profiles, and invalid Profile IDs to this return code?

Updated wording Co-authored-by: James Sandford <james.sandford@bbc.co.uk>

… flow-profiles

j616 · 2026-07-03T14:01:51Z

@@ -1,10 +1,10 @@
 {
    "title": "Audio Flow",
    "description": "Describes an audio Flow",


When rendered, these descriptions end up as the flow_metadata description for each type at the /profiles endpoint. I think this is unintended given the specific description in the profile schema

It's a little ugly, but the best way I could find to make this work is to move the description under the items in oneOf. I think it's because the description is tied to the selectable items, rather than the top level parameter.

j616 · 2026-07-03T14:24:57Z

+            "type": "string",
+            "pattern": "^[^\\s\/]+/[^\\s\/]+$"
+        },
+        "avg_bit_rate": 


I think avg_bit_rate and max_bit_rate either need re-locating such that they aren't in the profile, or we need to add notes to their specific CRUD endpoints stating they can't be modified where profile is set. iirc, they are update-able because they can't always be known when the Flow ingest is configured (e.g. when configuring a push-based live ingest sink) and they may vary over time in some cases.

For tracking purposes, after discussion the plan is as follows:

max_bit_rate will move back to being in the flow as this is something that should be measured at the point of content creation and since it's not fixed should not be in the profile

avg_bit_rate will remain in the profile as in practise this should be the same as the value the encoder is using as it's target bitrate. Also for profiles to work you need to specify the target bitrate to be able to distinguish between different bitrate versions of the same codec (eg 50Mbps v 100Mbps). This change does effectively move average to being an expectation, which is how it's really being used and achieves the aim of the field, so the ADR which talks about this is going to need updating

Wording needs to be added around what happens if someone tries to update a field in a flow which has come from a profile. It is expected that to do this the calling system will remove the profile_id by leaving it blank which will break the link to the profile as part of update message. Updates where the profile_id is still present SHOULD be rejected by the store as invalid with a suitable error message.

I've put in #236 to consider making the bitrate thing clearer overall in the spec

Co-authored-by: James Sandford <james.sandford@bbc.co.uk>

… flow-profiles

j616 · 2026-07-23T14:47:50Z

+When a Flow is created from a Profile then it inherits all the technical characteristics from the Profile.
+This means if that Flow then needs to be updated then consideration needs to be given to the fields which were inherited.
+
+For the standard fields (eg label, description, tags) which form part of the common Flow metadata and provided directly regardless of the Profile then these can be updated through the standard process.


I think this and the following paragraph need to be moved/copied to the top level Flow PUT endpoint's top level description (the one in the main YAML file) and the average bitrate PUT as it describes normative behaviour Service implementations should implement and Clients should expect. AppNotes are usually considered Informative/advisory.

johnbilt added 2 commits May 20, 2025 11:47

Flow profiles draft app note

0b0ab0a

Formatting improvements

b1150d3

johnbilt requested a review from a team as a code owner May 20, 2025 10:50

himslm01 reviewed Jun 3, 2025

View reviewed changes

Incorporating changes from Mark

d8680fe

j616 reviewed Jun 4, 2025

View reviewed changes

Comment thread docs/appnotes/00xx-using-flow-profiles.md Outdated

Comment thread docs/appnotes/00xx-using-flow-profiles.md Outdated

Comment thread docs/appnotes/00xx-using-flow-profiles.md Outdated

Comment thread docs/appnotes/00xx-using-flow-profiles.md Outdated

johnbilt and others added 9 commits June 4, 2025 17:26

Update docs/appnotes/00xx-using-flow-profiles.md

2a686d6

Updated wording Co-authored-by: James Sandford <james.sandford@bbc.co.uk>

Combining changes

c7a9347

Merge branch 'flow-profiles' of https://github.com/johnbilt/tams into…

3bb8bd0

… flow-profiles

Merge branch 'bbc:main' into flow-profiles

117b99e

Initial profile examples

1e71dbd

Merge branch 'bbc:main' into flow-profiles

3f1ccb7

Merge branch 'bbc:main' into flow-profiles

e89aa37

First set of profile schema changes

6770a5d

Merge branch 'flow-profiles' of https://github.com/johnbilt/tams into…

f741890

… flow-profiles

samdbmg reviewed Jul 31, 2025

View reviewed changes

GeorginaShippey reviewed Aug 5, 2025

View reviewed changes

j616 mentioned this pull request Aug 7, 2025

appnote: Add appnote covering Authorisation #115

Closed

11 tasks

johnbilt and others added 11 commits August 15, 2025 13:48

Update docs/appnotes/00xx-using-flow-profiles.md

d3a8c22

Co-authored-by: Sam Mesterton-Gibbons <sam@samn.co.uk>

Update api/schemas/flow-put.json

17e44f3

Co-authored-by: Sam Mesterton-Gibbons <sam@samn.co.uk>

Various updates from feedback

3d4f1f7

Merge branch 'flow-profiles' of https://github.com/johnbilt/tams into…

b6fc547

… flow-profiles

Update api/TimeAddressableMediaStore.yaml

f7899bb

Co-authored-by: Georgina Shippey <9370167+GeorginaShippey@users.noreply.github.com>

Update api/TimeAddressableMediaStore.yaml

4211448

Co-authored-by: Georgina Shippey <9370167+GeorginaShippey@users.noreply.github.com>

Correct example file name

d43ee0a

Merge branch 'flow-profiles' of https://github.com/johnbilt/tams into…

7b4a58e

… flow-profiles

Profile query clarification

e5655fe

Remove profile urn and add wording on multi-store workflows

49ee246

Updated app note number

7b51eb8

johnbilt added 8 commits May 12, 2026 11:32

Add comment on greedy match

980dd16

Merge remote-tracking branch 'forRepoRequest/main' into flow-profiles

233221a

Remove duplicate sections from merge

7af38cd

Correct app note numbering

ed1a948

General wording improvements

c9f13cd

Flow Profiles adr added

d3c4668

Lint fixes

27b7924

Fix validation fails

5437988

j616 reviewed May 13, 2026

View reviewed changes

johnbilt and others added 9 commits June 24, 2026 19:01

Update api/TimeAddressableMediaStore.yaml

2aea1c5

Updated wording Co-authored-by: James Sandford <james.sandford@bbc.co.uk>

Changes based on feedback from James S

d2cfef1

Merge branch 'flow-profiles' of https://github.com/johnbilt/tams into…

1eb6cdb

… flow-profiles

Added more video profile examples

98bbece

Updated wording

05d3ce0

Merge remote-tracking branch 'upstream/main' into flow-profiles

431234a

Fix lint line breaks

70f124a

Re-work replication to store where Profile does not exist

02421dc

Error on Profile does not exist

5f02623

j616 reviewed Jul 3, 2026

View reviewed changes

johnbilt and others added 9 commits July 21, 2026 12:40

Update api/TimeAddressableMediaStore.yaml

a17a9a8

Co-authored-by: James Sandford <james.sandford@bbc.co.uk>

Update api/schemas/flow-get.json

a90ed67

Co-authored-by: James Sandford <james.sandford@bbc.co.uk>

Update api/schemas/profile.json

4ce3fc7

Co-authored-by: James Sandford <james.sandford@bbc.co.uk>

Update api/TimeAddressableMediaStore.yaml

fcb7d07

Co-authored-by: James Sandford <james.sandford@bbc.co.uk>

Update api/TimeAddressableMediaStore.yaml

7f98b5f

Co-authored-by: James Sandford <james.sandford@bbc.co.uk>

Update api/schemas/flow-put.json

4187563

Co-authored-by: James Sandford <james.sandford@bbc.co.uk>

Profiles query format updated to remove multi

b9d377c

Merge branch 'flow-profiles' of https://github.com/johnbilt/tams into…

6f5da61

… flow-profiles

Changes to bitrate fields

2ee1780

j616 requested changes Jul 23, 2026

View reviewed changes

samdbmg mentioned this pull request Jul 23, 2026

Be clearer about how the bitrate fields in Flows should be set #236

Open

	Supplying a Profile ID and technical metadata should result in a 400 validation error
	Supplying a Profile ID and technical metadata defined within the specified Profile should result in a 400 validation error

Uh oh!

Conversation

johnbilt commented May 20, 2025 • edited by j616 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Details

Jira Issue (if relevant)

Related PRs

Submitter PR Checks

Reviewer PR Checks

Info on PRs

Uh oh!

himslm01 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

samdbmg left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

samdbmg commented Jul 31, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

johnbilt commented May 20, 2025 •

edited by j616

Loading