When I started programming with Python (a switch I would urge any casual utility programmer to make), I learned basic syntax. I very very quickly learned that if you’re ever wondering if there’s the proverbial “better way” to do something, then there is probably already a utility built into Python to accomplish the task.

Consider the ‘super’ functionality in Python: when I first encountered this, I had no idea what it was or what it meant. Furthermore, the documentation on it is very obscure and doesn’t seem to clarify anything about what it does. I Google’d it (of course) and instantly found results with titles like “Python’s Super Considered Harmful” and “Problems with the new super()“. Encouraging.

It wasn’t until I began working in Django, a pure Python-based web framework (which I would highly recommend, if you’re in the web business: http://www.djangoproject.com/ ), when I was reading over others’ source code that I realized what you could do with super.

If you have a class A,which defines some method called routine, like so:


class A(object): # In Python 3, you can just do class A:
    def routine(self):
        print "A.routine()"

And then a class B, which inherits from A, which also defines a routine method:


class B(A):
    def routine(self):
        print "B.routine()"

We instantiate with this:

>>> b = B()

For simplicity, this example is stupidly simple. Now, if you instantiate B, and call it’s only method, you’ll only ever be running B‘s routine method, and no matter many times you call it it’ll never give you that of class A.

Now consider if class B had the added a line to the end of it’s method:


class B(A):
    def routine(self):
        print "B.routine()"
        super(B, self).routine() # In Python 3, you can just do super().routine()

In this case, ‘super’ will return the A object which is underneath B.  You give super() a handle to its own class, and then an actual instance of that same class.  Hence, we gave it “B”, and then “self”.  Super returns the literal parent of the active B object (which is the local variable ‘b’, because we passed it ‘self’).  It is not returning the simple generic class; instead, it is returning the actual A which was created when the local variable b was created.  This is called a “bound” class object, because it’s referring to an actual parent class object in memory, instead of just the class blueprint.

This is what happens when we create a new B object now:

>>> b = B()
>>> b.routine()
'B.routine()'
'A.routine()'

Simply put, this kind of usage of the super method is often used to “pass control up” to the parent class, after the subclass intercepts data.

Finally, if you’re interested, here is a more practical example:

from some.package import A
# We don't need to know anything about the base class "A",
# but we do want to intercept one of its parameters, "foo",
# and change it, sending it along as before.
# This new class could be used in place of "A" without
# other sections of code even knowing the difference.
class MyClass(A):
    def render(self, foo, *args, **kwargs):
        ''' this receives a var named 'foo', a tuple of
        unnamed 'args',and a dictionary of named 'kwargs' '''
        # Append a quick prefix to the variable 'foo'
        foo = "intercepted by MyClass - " + foo
        super(MyClass, self).render(foo, *args, **kwargs)

In this example, we don’t need to know anything about class A, except for the fact that we want to alter the variable ‘foo’ when it comes into A‘s render method.  Note that ‘*args’ catches any unnamed arguments passed to MyClass, and that ‘**kwargs’ is the common abbreviation for ‘key-word arguments’.

Also note that the only reason why MyClass‘s render method ALSO takes bunches of arguments is because we model it to look exactly like A‘s render method.  We want MyClass to seamlessly integrate with some other code.  That other code should never have a reason know the difference between A and MyClass.

All this does is change ‘foo’, and then passes control back up to the parent class A, where the data was intended to go.  We cleanly call the super method, which returns A, with all of its unknown methods and fields.  We then call ‘render‘ on that returned object, in order to execute A‘s own render method (and not our overloaded one in MyClass).

By passing A its arguments with those prefixing * characters, we preserve how they were passed into myClass.  Keyword arguments get turned into a dictionary while in MyClass.render, but A.render wants them as keyword arguments still, not a dictionary.  So, we use the dereferencing * characters to turn it back into keyword arguments.

Clean, huh?  This is extremely common in Django code, because Django gives you base classes to model from.  You then have the power to easily overload those model methods, do some custom task, and then pass control back up to the model’s method for the intended behavior.

While super is nice, it only resolves into a single parent class, such that multiple inheritance (where multiple parent classes have the same method name) won’t know how to decide between which method to run.  Instead, you can directly invoke the parent class’s method in a more manual manner, such as “SecondParentClass.render(self, foo, *args, **kwargs)”.  Note that you pass a reference to ‘self’ in that method call, to properly put things into scope.

Tagged with:
 

43 Responses to Understanding Python’s ‘super’

  1. Matthias says:

    There are several errors here:

    First, A should inherit from ‘object’ (super works only for new-style classes). Second the first argument to the super function should be B and not A. You want to delegate your call to the parent class, so it should be:

    super( B, self ).routine()

    Matthias

    • The first thing you say, about only working on new-style classes, is true, but my example is working from a simplified hypothetical demonstration. Hence, I anonymized the classes to simple “A” and “B”. What you said is correct, but my example wasn’t illustrating the process from its absolute beginning; only the application of the process.

      Second thing, about the argument being B and not A, you are absolutely correct. I had typed this simply from memory, and clearly I mixed it up. I will make the fix in just a moment.

      Tim

  2. dezza says:

    If you want your examples to work in Python’s later than (2.2) you have to inherit your A class from object.

    class A(object):

    instead of just

    class A:

    Some really good sources for understanding super completely is these two:

    http://www.cafepy.com/article/python_attributes_and_methods/python_attributes_and_methods.html#method-resolution-order

    http://www.python-forum.org/pythonforum/viewtopic.php?f=3&t=15649#p72191

    • Yes, I am aware. I was trying to be purely abstract at the time of writing, but I can see how this could be misleading to someone trying to make use of the example.

      I’ll update it to reflect this feedback. One of the earlier comments said something similar.

  3. AP says:

    “Just imagine what kind of power you would wield if you had multiple inheritance going on, where you could decide at runtime which parent class to use the super method on.”

    When you call super, python returns the next object in the MRO, so you can’t decide what parent class it will return. It may not even return a parent class, as is clearly explained here: http://fuhm.net/super-harmful/

    • Tim says:

      You’re correct, if you were just calling super() like this. I didn’t explain myself very well in this post, back when I wrote it, but I had been thinking along the lines of testing inheritance of an object with “isinstance” (using the ‘abc’ interface framework or just normal class hierarchy), and then circumventing ‘super()’ in favor of just calling the method straight off the desired parent class. This syntax isn’t as nice as super(), but it’s how things were done before super() really got mileage.

  4. Henry van der Beek says:

    I don’t know if it’s just because I’m viewing in Linux, but your code doesn’t seem to have any tabs in it. It’s quite hard to understand what level you are performing the functions on.

    • Tim says:

      WordPress likes to strip out the “unnecessary” spaces. I’ll try to fix it again. Any time I edit the post, it loses the spaces again.

      • Liang says:

        Try Octopress. It’s better than wordpress. With Ocotpress, you could use markdown, which will handle spaces well. (I’m not advertising — just give your a suggestion.)

        • Tim says:

          I actually loaded up a markdown plugin for wordpress a while back. I need to comb through all of those old posts and fix them up still.

  5. andi says:

    I ‘ve been looking for the ‘super’ information all over the net, after read couple of article about ‘super’, and today i found this article, guess what…… this is the most understandable articel i’ve found so far! Cool man LOL.

  6. Michael says:

    Thanks – makes sense now.

  7. Chris Lasher says:

    The indentation on your code examples is completely hosed at the moment.

  8. kay-python says:

    Hey, thanks for this post. I just started a new job and the codebase has quite a few instances of “super”. So this has been extremely helpful. The “super” function is very useful now that i understand it. thanks again

  9. Everest says:

    Wow! such a simple and clear explanation!!! Myself learning Django too.It was cool and easy to understand. There is plenty of stuff in the net in any topic, but simplicity and easy to understand explanation is very difficult to find. I found yours after spending some time on many other places in the internet and books. You simply rock!!. Please keep on posting such useful tips related to Python/Django or any other things in such a simple and clear way.

    Many Thanks.

  10. Mike A says:

    Nice clear explanation. Needed to explain it to someone, but it has been a long time since I’ve written any python classes using inheritance. So glad that I got rid of that awful habit.

    • Tim says:

      It’s a tool, to be used within the bounds of sane limits. It’s pretty much just class Interfaces but with non-abstract default code, so it’s not that awful.

  11. Seren Seraph says:

    So why on earth is it super(CurrentClass, self).whatever(..) when CurrentClass is derivable from self? I imagine it is so I can get super relative to some other class up my inheritance graph, right? But then I would have to know the class whose super I really wanted. How exactly is this easier than ClassIKnowIWant.whatever(self,..)? I guess it may be easier if I am not sure exactly what in my inheritance graph implements ‘whatever’ but I know I want which everyone is closest to the the current class in the inheritance graph. *scratches head*

    • Tim says:

      In the case of single-inheritance, you’re correct. But in classes that are derived from multiple bases, super() is giving you the benefit of not having to explicitly name the base class that will ultimately handle the method invokation. This is great for “mix-in” classes or ones where the MRO (method resolution order) decides to override one inherited method with another.

      Ultimately, though, you are right about the redundant naming of the current class– it doesn’t solve the problem of maintainability where you don’t want to repeat yourself too much. So, starting with the Python 3.x series (and I thought it was backported to Python 2.7, but I could be wrong about that) you can use super() with no parameters, and it’ll do the same thing. If you need to specifically name the parameters, you are still free to do so, however.

  12. Aaron Newton says:

    Thank you for that.

    I had read several explanations on StackOverflow (strangely enough I was also researching this for the purposes of understanding something in Django), but your explanation was actually the easiest to understand so far.

  13. Justin K says:

    Your explanation of super() really helped me. Your formatting and wording is very easy to understand. Thank you.

  14. Ted says:

    Thanks for this. Cleanest and simplest explanation of ‘super’ that I have come across, and I have done a lot of googling and reading. This really helps me understand better django code that I am leveraging for my project.

  15. Rob B says:

    Great explanation of super() Tim, much better than the official docs. Thanks!

  16. John says:

    Thanks, the cleanest explanation I have had yet of super(), it was very helpful

  17. Dilip says:

    Cool! Nice explanation. Thanks.

  18. Andreas Kostyrka says:

    Actually, super() does work for multiple inheritence and it was designed to solve the problem of cooperating methods in complex class diagrams.

    For single inheritence, naming directly the base class (BaseClass.routine(self)) is certainly the easier and way simpler solution.

    Another thing you didn’t mention is that if you use super, all classes need to use super (because super() accesses the class next in the MRO, which is not necessarily your base class, and in this case calling naively your base class can result in ugly infinite recursion.

    Last but not least, you did not mention that super() requires a very delicate argument handling. (Basically either all your methods have exactly the same arguments, or you need to use keyword arguments.) That follows from the fact that super() is explicitely used to decide which class to call next, which might be not a base class at all, it might be a sibling class. So either your method takes a fixed argument format, or it needs to be able to handle a generic argument set (**kwargs)

    • Tim says:

      Hi, I wanted to say that while I’m aware of your correct observations, the purpose of my post is less about providing a full technical description of how super() works, and more about explaining wtf it’s even used for, so that people unfamiliar with the matter can grasp at its goal.

      Ultimately, this is the reason why man pages are not helpful for so many people–they include all the grimy details, taking zero consideration for the fact that someone unfamiliar with the tool doesn’t even understand what some of the vocabulary even means.

      My goal here was to provide a simpler explanation, making no assumptions about a person’s existing technical knowledge on the subject of MRO. If a person wants the unfiltered truth about super(), they can do a search for it and 100% of all other resources will get brutally technical and use words and phrases that read like a research paper.

      Consequently, this is not the role my post is fulfilling. I wrote it because the bloody thing was not easy to comprehend when years ago I first discovered it in others’ code, and when I searched for it’s meaning, the explanations appeared to assume that I already knew what it did, and were thus quite unclear.

      We’ve all been there–at a friend’s house about to play a card game they you’ve never heard of. 6 people try to explain the rules simultaneously, all doing a horrible job at simplifying and telling you only what you need to know so that you can form your own understanding. Sometimes it is a lot easier to just start with the basics.

      When a person gets to the point in their programming adventures when they ask a question like “Wow, how do I do this with multiple classes?” or “What if I have more than one parent class with a method using same name?”, they will be able to find their answers–not because of this post, but because they formed the question on their own.

  19. dan says:

    just saying thanks for the explanation. i was confused at first by some of the comments, but then realized you had already gone back to correct. thanks!!!

  20. dan says:

    maybe a dumb question, but in the MyClass example, when you call the super, shouldnt the M in myClass be capitalized? im not sure, but i thought that it was meant to match the name of the new class, and not just some new parameter you can make up (like in for loops)

    • Tim says:

      Yep yep. Good catch. I tried to modernize the code examples a couple of months ago, and I’m slowly working on a rewrite of the post, because of how much traffic it gets. Hopefully I can update it without screwing up formatting :D

  21. […] super(AssignManagerForm, self).__init__(*args, **kwargs)  calls the __init__ method of parent class, which is the Form class in this case. This following blog post describes super really well:http://blog.timvalenta.com/2009/02/understanding-pythons-super/. […]

  22. Davi says:

    Did you mean override, not overload (and overidden, not overloaded)?
    Also, ">>>" is displayed instead of ">>>".
    Thanks for the education.

  23. Davi says:

    Oops, that didn’t work. Try again:
    Also, ">>>" is displayed instead of ">>>".

    • Tim says:

      Now you see the problem sometimes :) WordPress drives me nuts. The smart editor wants to watch the world burn. No matter how many times I fix it, it reverts back to encoded entities when I update the post.

  24. yevheniyc says:

    Thank you!

  25. yevheniyc says:

    Thank you for clarifying super() for me. Check out Django Tutorial by Mike Hibbert if interested learning about Django. Great framework for Python! I just went through all of the tutorials myself.

  26. Bartek says:

    I’ve read a bunch of posts on python’s super(), but it’s for the first time I’m feeling like I just got it. Thanks buddy!

  27. Jack says:

    I love this… “For simplicity, this example is stupidly simple.” I would be offended if it weren’t for the fact that was exactly what I need when I am trying to grasp this. Been trying to thoroughly understand super() for weeks. Thanks tons! You just unlocked a large chunk of Django for me.

  28. […] Understanding Python’s super […]

  29. Ahmed Sadman says:

    Ah! at last. I was searching for this kind of easy explanation.

  30. Francis says:

    Thanks for your simple concise explanation. Atleast now I have a better idea what super() is. A lot better than the official python guide and others out there.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>