Add support for source with attributes in extra_kwargs of ModelSerializer by BergLucas · Pull Request #9077 · encode/django-rest-framework

BergLucas · 2023-08-18T07:59:40Z

Description

Hello dear maintainers, it's my first contribution to django rest framework so I hope I didn't do anything wrong.

In the pull request referenced above, there is someone that mentioned that we could not use source with attributes in extra_kwargs of a ModelSerializer.

For example, the following code would create an error:

class MyUser(models.Model):

    user = models.OneToOneField(User, on_delete=models.DO_NOTHING)

class MyUserSerializer(serializers.ModelSerializer):

    class Meta:
        model = MyUser
        fields = (
            "username",
        )
        extra_kwargs = {
            "username": {"source": "user.username"},
        }

I think it could be a very interesting feature because at the moment, when we have a foreign key to another model, we're obliged to specify the fields explicitly in the serializer. However, this means that if we have special validators on the fields of our model, we're obliged to put them back on the serializer fields.

For example, the username field of Django's default User has a special validator so if we just define a basic CharField, it would not validate the data the same way as the model would validate it:

class MyUser(models.Model):

    user = models.OneToOneField(User, on_delete=models.DO_NOTHING)

class MyUserSerializer(serializers.ModelSerializer):

    username = serializers.CharField(source="user.username")

    class Meta:
        model = MyUser
        fields = (
            "username",
        )

This could create a difference between the way the model validates data and the way the serializer validates data if we forgot a validator or if we change the model without changing the serializer.

In this pull request, I added this feature so that we could just specify the model field we want in the source of extra_kwargs and it will generate the right field on the serializer.

The code may seem a little odd, but I've tried to keep the changes in one place only. I've also tried to maintain a good error message so that, if at any point the path to the field is wrong, it returns the full path in the error message and not just part of it.

Thanks for reading and please let me know if there are any changes that could be made.

auvipy

could you please add proper test cases to validate the changes proposed?

auvipy · 2023-09-14T13:31:40Z

___________________________________ summary ____________________________________
ERROR: py36-django30: commands failed
py36-django31: commands succeeded
py36-django32: commands succeeded

auvipy

that was a nice example

auvipy

And this should also require documentation update as far as I can understand!

BergLucas · 2023-09-14T16:37:01Z

Hi @auvipy,

And this should also require documentation update as far as I can understand!

Do you mean to add a section here (or elsewhere) to document the feature?

auvipy · 2023-10-02T05:47:53Z

Hi @auvipy,

And this should also require documentation update as far as I can understand!

Do you mean to add a section here (or elsewhere) to document the feature?

yup there or somewhere else where it would be relevant

auvipy

does this change conform with the principle of adding compatibility with django? I'm not fully convinced about the merit of the new change

stale · 2025-06-27T07:36:38Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

ulgens · 2025-06-27T09:23:40Z

I think this is still relevant and shouldn't be marked as stale.

peterthomassen · 2025-06-27T09:23:34Z

+                    if attr not in attr_info.relations:
+                        break


This seems to cause trailing attributes to be ignored if they are "after" other relation attributes, but are not a relation themselves. Is this intended?

If so, I think this scenario should also have a test case.

It is not really intended. Is it possible to have attributes other than relations with sufficient information in this function to be able to follow them? As far as I know, this isn't the case, so I just did it this way. It's also possible that I misunderstood your comment

ulgens · 2025-06-27T09:32:06Z

@auvipy Do you know why the bot doesn't remove stale label when there is a new comment?

ulgens · 2025-07-18T11:57:39Z

@BergLucas Hey 👋🏻 Is it possible for you to rebase this branch from the latest main, so it won't be closed as stale and we can push for another tour of review & testing? Thanks in advance.

…://github.com/BergLucas/django-rest-framework into improvement/source-attributes-in-extra_kwargs

BergLucas · 2025-07-19T13:39:25Z

Hi @ulgens ,

Thanks for your reviews! I've rebased the branch and responded to the comments.

(Sorry for the delay, I've been quite busy these last few weeks.)

Copilot

Pull request overview

Adds support for using dotted source paths (e.g. user.username) in ModelSerializer.Meta.extra_kwargs so that DRF can infer the correct serializer field type/validators from related model fields, without requiring explicit field declarations.

Changes:

Update ModelSerializer.get_fields() to resolve dotted source paths across model relations when building fields.
Add a regression test asserting the generated field representation includes validators from related model fields (e.g. User.username).
Document the new extra_kwargs capability with an example mapping serializer fields to related model attributes.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

File	Description
`rest_framework/serializers.py`	Adds dotted-source traversal to choose the correct related model metadata when building auto-generated fields.
`tests/test_model_serializer.py`	Adds coverage for dotted `source` paths in `extra_kwargs` and expected field mapping/validators.
`docs/api-guide/serializers.md`	Documents using `extra_kwargs` with `source` to map serializer fields to related model fields.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-02T14:33:11Z

+                if attr not in attr_info.relations:
+                    break
+
+                attr_model = attr_info.relations[attr].related_model


The dotted source traversal rewrites the field based on related model metadata, but it currently follows to-many relations too (e.g. M2M or reverse FK). That can generate a scalar field for sources like groups.name, which will then fail/behave incorrectly at runtime because attribute traversal hits a RelatedManager.

Consider bailing out when an intermediate relation has to_many=True (and keep source unchanged so build_field raises ImproperlyConfigured), or raise a clearer configuration error for dotted sources that traverse collections.

Suggested change

if attr not in attr_info.relations:

break

attr_model = attr_info.relations[attr].related_model

relation_info = attr_info.relations.get(attr)

if relation_info is None:

break

if getattr(relation_info, 'to_many', False):

# Do not rewrite sources that traverse to-many relations.

break

attr_model = relation_info.related_model

Copilot · 2026-04-02T14:33:11Z

+            source_attrs = source.split('.')
+            source_info = info
+            source_model = model
+
+            attr_info = info
+            attr_model = model
+
+            for attr in source_attrs[:-1]:
+                if attr not in attr_info.relations:
+                    break
+
+                attr_model = attr_info.relations[attr].related_model
+                attr_info = model_meta.get_field_info(attr_model)
+            else:
+                attr = source_attrs[-1]
+                if (
+                    attr in attr_info.fields_and_pk
+                    or attr in attr_info.relations
+                    or hasattr(attr_model, attr)
+                    or attr == self.url_field_name
+                ):
+                    source = attr
+                    source_info = attr_info
+                    source_model = attr_model


This logic runs for every auto-generated field, even when source has no dotted path. You can avoid the extra split()/loop and repeated get_field_info() calls by guarding the new traversal with something like if '.' in source: (or if len(source_attrs) > 1: after splitting).

Suggested change

source_attrs = source.split('.')

source_info = info

source_model = model

attr_info = info

attr_model = model

for attr in source_attrs[:-1]:

if attr not in attr_info.relations:

break

attr_model = attr_info.relations[attr].related_model

attr_info = model_meta.get_field_info(attr_model)

else:

attr = source_attrs[-1]

if (

attr in attr_info.fields_and_pk

or attr in attr_info.relations

or hasattr(attr_model, attr)

or attr == self.url_field_name

):

source = attr

source_info = attr_info

source_model = attr_model

source_info = info

source_model = model

if '.' in source:

source_attrs = source.split('.')

attr_info = info

attr_model = model

for attr in source_attrs[:-1]:

if attr not in attr_info.relations:

break

attr_model = attr_info.relations[attr].related_model

attr_info = model_meta.get_field_info(attr_model)

else:

attr = source_attrs[-1]

if (

attr in attr_info.fields_and_pk

or attr in attr_info.relations

or hasattr(attr_model, attr)

or attr == self.url_field_name

):

source = attr

source_info = attr_info

source_model = attr_model

Copilot · 2026-04-02T14:33:12Z

+                'first_name': {'source': 'user.first_name'},
+                'last_name': {'source': 'user.last_name'}


The new example uses dotted sources to pull fields from the related user. By default, ModelSerializer.create()/update() does not support writable dotted-source fields (it asserts unless you set read_only=True or implement explicit create/update handling). It’d help to either mark these example fields as read_only=True (via extra_kwargs) or add a short note clarifying the write behavior.

Suggested change

'first_name': {'source': 'user.first_name'},

'last_name': {'source': 'user.last_name'}

'first_name': {'source': 'user.first_name', 'read_only': True},

'last_name': {'source': 'user.last_name', 'read_only': True},

Copilot · 2026-04-02T14:33:12Z

+

The test covers the happy path for dotted sources, but it doesn’t cover an important failure mode introduced by the traversal: dotted sources that cross a to-many relation (M2M or reverse FK) should raise ImproperlyConfigured instead of generating a field that can’t be resolved at runtime. Adding an assertion test for a source like user.groups.name (or any to-many hop) would prevent regressions here.

Suggested change

def test_source_with_to_many_raises_improperly_configured(self):

class UserProfile(models.Model):

user = models.ForeignKey(User, on_delete=models.CASCADE)

class InvalidUserProfileSerializer(serializers.ModelSerializer):

groups = serializers.CharField(source='user.groups.name')

class Meta:

model = UserProfile

fields = ('groups',)

with self.assertRaises(ImproperlyConfigured):

InvalidUserProfileSerializer()

Add support for source with attributes in extra_kwargs

dd0bb97

auvipy self-requested a review August 19, 2023 15:05

auvipy requested changes Aug 21, 2023

View reviewed changes

Add test case for source with attributes in extra_kwargs

705e277

auvipy requested review from a team and auvipy August 23, 2023 14:25

auvipy requested changes Sep 3, 2023

View reviewed changes

Comment thread tests/test_model_serializer.py Outdated

Add a real world example

c1ccc37

auvipy requested changes Sep 14, 2023

View reviewed changes

Comment thread tests/test_model_serializer.py Outdated

auvipy reviewed Sep 14, 2023

View reviewed changes

auvipy added this to the 3.15 milestone Sep 14, 2023

auvipy requested changes Sep 14, 2023

View reviewed changes

BergLucas force-pushed the improvement/source-attributes-in-extra_kwargs branch from b10dfc2 to c1ccc37 Compare September 14, 2023 15:36

Fix test for Django 3.0

a5b239e

auvipy requested a review from a team October 2, 2023 05:52

Add some documentation about source in extra_kwargs

2fc08b2

auvipy reviewed Apr 27, 2025

View reviewed changes

Comment thread tests/test_model_serializer.py Outdated

auvipy removed this from the 3.15 milestone Apr 27, 2025

BergLucas added 2 commits April 27, 2025 22:47

Merge branch 'master' into improvement/source-attributes-in-extra_kwargs

05b2e3b

Change the test "test_source_with_attributes" to target django 4.2-5.2

b4ce3e5

auvipy requested a review from a team April 28, 2025 03:26

stale Bot added the stale label Jun 27, 2025

peterthomassen requested changes Jun 27, 2025

View reviewed changes

stale Bot removed the stale label Jul 18, 2025

BergLucas added 8 commits July 19, 2025 14:53

Add support for source with attributes in extra_kwargs

53141ac

Add test case for source with attributes in extra_kwargs

d60b909

Add a real world example

84fd5ee

Fix test for Django 3.0

5be47e4

Add some documentation about source in extra_kwargs

5a106a0

Change the test "test_source_with_attributes" to target django 4.2-5.2

af69a58

Merge branch 'improvement/source-attributes-in-extra_kwargs' of https…

998025a

…://github.com/BergLucas/django-rest-framework into improvement/source-attributes-in-extra_kwargs

Remove useless if

f15196a

auvipy requested review from auvipy and Copilot April 2, 2026 14:27

Copilot started reviewing on behalf of auvipy April 2, 2026 14:29 View session

Copilot AI reviewed Apr 2, 2026

View reviewed changes

-                if attr not in attr_info.relations:
-                    break
-                attr_model = attr_info.relations[attr].related_model
+                relation_info = attr_info.relations.get(attr)
+                if relation_info is None:
+                    break
+                if getattr(relation_info, 'to_many', False):
+                    # Do not rewrite sources that traverse to-many relations.
+                    break
+                attr_model = relation_info.related_model

		'first_name': {'source': 'user.first_name'},
		'last_name': {'source': 'user.last_name'}

+    def test_source_with_to_many_raises_improperly_configured(self):
+        class UserProfile(models.Model):
+            user = models.ForeignKey(User, on_delete=models.CASCADE)
+        class InvalidUserProfileSerializer(serializers.ModelSerializer):
+            groups = serializers.CharField(source='user.groups.name')
+            class Meta:
+                model = UserProfile
+                fields = ('groups',)
+        with self.assertRaises(ImproperlyConfigured):
+            InvalidUserProfileSerializer()

Uh oh!

Conversation

BergLucas commented Aug 18, 2023

Description

Uh oh!

auvipy left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

auvipy commented Sep 14, 2023

Uh oh!

auvipy left a comment

Choose a reason for hiding this comment

Uh oh!

auvipy left a comment

Choose a reason for hiding this comment

Uh oh!

BergLucas commented Sep 14, 2023

Uh oh!

auvipy commented Oct 2, 2023

Uh oh!

auvipy left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

stale Bot commented Jun 27, 2025

Uh oh!

ulgens commented Jun 27, 2025

Uh oh!

peterthomassen Jun 27, 2025

Choose a reason for hiding this comment

Uh oh!

BergLucas Jul 19, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ulgens commented Jun 27, 2025

Uh oh!

ulgens commented Jul 18, 2025

Uh oh!

BergLucas commented Jul 19, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Apr 2, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 2, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 2, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 2, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

auvipy left a comment •

edited

Loading