After reading the title, you probably ask yourself something along the lines of, “Functions in Python are an advanced concept? How? All courses introduce functions as the basic block in language.” And you are right and wrong at the same time.

Most courses on Python introduce functions as a basic concept and building block because, without them, you would not be able to write functional code at all. This is totally different from the functional programming paradigm, which is a separate concept, but I shall touch upon this one too.

Before we delve into the advanced intricacies of Python functions, let’s briefly go over some basic concepts and things you probably already know.

Brief basics

So you start writing your program, and at some point you end up writing the same sequence of code. You start repeating yourself and the code blocks. This proves to be a good time and place to introduce functions. At least, that is. In Python, you define a function as:

def shout(name):
    print(f'Hey! My name is {name}.') 

In the world of software engineering, we make a distinction between parts of function definition:

• def - Python keyword used to define a function.
• shout - function name.
• shout(name)- function declaration.
• name - function argument.
• print(...)is a part of a function body or how we call it function definition.

A function can return a value or have no return value at all, like the one we previously defined. When function returns value it can return one or more of them:

def break_sentence(sentence):
    return sentence.split(' ')

What you get as a result is a tuple that you can unpack or pick any of the tuple elements to proceed with.

For those of you who are not informed, functions in Python are first-class citizens. What does this mean? It means you can work with functions as you would with any other variable. You can pass them as arguments to other functions, return them from functions, and even store them in variables. Here is one of the examples:

def shout(name):
    return f'Hey! My name is {name}.'

# we will use break_sentence defined above

# assign function to another variable
another_breaker = break_sentence 

another_breaker(shout('John'))
# ['Hey!', 'My', 'name', 'is', 'John.']

# Woah! Yes, this is a valid way to define function
name_decorator = lambda x: '-'.join(list(name))

name_decorator('John')
# 'J-o-h-n'

Wait, what was this lambda ? This is another way you can define functions in Python. This is the so-called unnamed or anonymous function. Well, in this example, we are assigning it to a variable named name_decorator but you can pass the lambda expression as an argument of another function without the need to name it. I will cover this shortly.

What is left is to give an example of how functions can be passed as arguments or returned as values from another function. This is the part where we are moving toward advanced concepts, so bear with me.

def dash_decorator(name):
    return '-'.join(list(name))

def no_decorator(name):
    return name

def shout(name, decorator=no_decorator):
    decorated_name = decorator(name)
    return f'Hey! My name is {decorated_name}'

shout('John')
# 'Hey! My name is John'

shout('John', decorator=dash_decorator)
# 'Hey! My name is J-o-h-n'

So this is how it looks to pass functions as arguments to another function. What about the lambda function? Well, take a look at the next example:

def shout(name, decorator=lambda x: x):
    decorated_name = decorator(name)
    return f'Hey! My name is {decorated_name}'

print(shout('John'))
# Hey! My name is John

print(shout('John', decorator=dash_decorator))
# Hey! My name is J-o-h-n

Now the default decorating function is lambda and returns the argument’s value as it is (idempotent). Here, it is anonymous because there is no name attached to it.

Notice that print is also a function, and we are passing a function shout inside of it as an argument. In essence, we are chaining functions. And this can lead us to a functional programming paradigm, which is a path that you can choose in Python. I will try to write another blog post specifically on this subject because it is very interesting to me. For now, we will keep to the procedural programming paradigm; that is, we will continue with what we have been doing so far.

As stated previously, a function can be assigned to a variable, passed as an argument to another function, and returned from that function. I have shown you some simple examples for the first two cases, but what about returning a function from a function? At first I wanted to keep it really simple, but then again, this is an Advanced Python blog post series!

Intermediate or advanced parts

This will by no means be THE guide to functions and advanced concepts around functions in Python. There are a lot of great materials, which I will leave at the end of this post. However, I want to talk about a couple of interesting aspects that I have found to be very intriguing.

Functions in Python are objects. How can we figure this out? Well, each object in Python is an instance of a class that eventually inherits from one specific class called type. The details of this are convoluted, but to be able to see what this has to do with functions, here is an example:

type(shout)
# function

type(type(shout))
# type

When you define a class in Python, it automatically inherits the object class. And which class does object inherit?

type(object)
# type

And should I tell you that classes in Python are objects too? Indeed, this is mind-boggling for beginners. But as Andrew Ng would say, this is not that important; don’t worry about it.

Okay, so functions are objects. Certainly functions should have some magic methods, then, right?

shout.__class__
# function

shout.__name__
# shout

shout.__call__
# <method-wrapper '__call__' of function object at 0x10d8b69e0>
# Oh snap!

The magic method __call__ is defined for objects that are callable. So our shout object (function) is callable. We can call it with or without arguments. But this is interesting. What we did previously was define a shout function and get an object that is callable with the __call__ magic method that is a function. Have you ever watched the Inception movie?

So, our function is not really a function but an object. Objects are instances of classes and contain methods and attributes, right? This is something you should know from OOP. How can we find out what the attributes of our object are? There is this Python function called vars that returns a dictionary of object attributes with their values. Let’s see what happens in the next example:

vars(shout)
# {}

shout.name = 'Jimmy'

vars(shout)
# {'name': 'Jimmy'} 

This is interesting. Not that you could figure out the use case for this straight away. And even if you could find it, I would highly discourage you from doing this black magic. It’s just not easy to follow, even though it is an interesting flex. The reason I have shown you this is because we wanted proof that functions are indeed objects. Remember, everything in Python is an object. That is how we roll in Python.

Now, long-awaited functions are returning. This concept is also very interesting since it gives you a lot of utility. With a little bit of syntactic sugar, you get very expressive. Let’s dive in.

First, a function’s definition can contain another function’s definition. Even more than one. Here is a perfectly fine example:

def shout(name):
    def _upper_case(s):
        return s.upper()

    return _upper_case(name)

If you are thinking that this is just a convoluted version of name.upper() you are right. But wait, we are getting there.

So, given the previous example, which is fully functional Python code, you can experiment with multiple functions defined inside your function. What is the value of this neat trick? Well, you could be in a situation where your function is huge with repeating blocks of code. This way, defining a subfunction would increase readability. In practice, huge functions are a sign of code smell, and it is highly encouraged to break them into a few smaller ones. So, following this advice, you will rarely have the need to define multiple functions inside each other. One thing to notice is that the _upper_case function is hidden and out of reach in the scope where the shout function ends up being defined and available to call. This way, you can’t test it easily, which is yet another issue with this approach.

However, there is one specific case where defining a function inside another is a way to go. This is when you implement the decorator of a function. This has nothing to do with the function we used to decorate the name string in one of previous examples.

Decorators in Python

What is a decorator function? Think of it as a function that wraps your function. The goal of doing this is to introduce additional functionality to an already existing function. For example, say you want to log every time your function is called:

def my_function():
    return sum(range(10))

def my_logger(fun):
    print(f'{fun.__name__} is being called!')
    return fun

my_function()
# 45

my_logger(my_function)
# my_function is being called!
# <function my_function at 0x105afbeb0>

my_logger(my_function)()
# my_function is being called!
# 45

Pay attention to how we decorate our function; we pass it as an argument to the decorating one. But this is not enough! Remember, decorator returns function, and this function needs to be invoked (called). This is what the last call does.

Now, in practice, what you really want is for decoration to persist under the original function’s name. In our case, we would like that after the interpreter parses our code,  my_function is the name of the decorated function. This way, we keep things simple to follow, and we are making sure that any part of our code won’t be able to call an undecorated version of our function. Example:

def my_function():
    return sum(range(10))

def my_logger(fun):
    print(f'{fun.__name__} is being called!')
    return fun

my_function = my_logger(my_function)

my_function(10)
# my_function is being called!
# 45

You will admit that the part where we reassign the function’s name to a decorated one is troublesome. You have to keep this in mind. If there are many function calls you want to log, there will be a lot of repeating code. Here is where the syntactic sugar comes in. After the decorator function is defined, you can use it to decorate the another function by prefixing the function definition with an @ and the name of the decorator function. Example:

def my_logger(fun):
    print(f'{fun.__name__} is being called!')
    return fun

@my_logger
def my_function():
    return sum(range(10))

my_function()
# my_function is being called!
# 45

This is Python’s Zen. Look at the expressiveness of the code and its simplicity.

One important thing to note here! Even though the output makes sense, it is not what you would expect! At the time of loading your Python code, the interpreter will call the my_logger function and effectively run it! You will get the log output, yet this will not be what we wanted in the first place. Look at the code now:

def my_logger(fun):
    print(f'{fun.__name__} is being called!')
    return fun

@my_logger
def my_function():
    return sum(range(10))

my_function()
# my_function is being called!
# 45
my_function()
# 45

To be able to run decorator code once the original function is called, we have to wrap it around another function. This is where things can get messy. Here is an example:

def my_logger(fun):
    def _inner_decorator(*args, **kwargs):
        print(f'{fun.__name__} is being called!')
        return fun(*args, **kwargs)

    return _inner_fun

@my_logger
def my_function(n):
    return sum(range(n))

print(my_function(5))
# my_function is being called!
# 10

In this example, there are some updates as well, so let’s go over them:

1. We want to be able to pass the argument to my_function.
2. We want to be able to decorate any function, not just my_function. Because we don’t know the exact number of arguments for future functions, we have to keep things as general as we can, which is why we use *args and **kwargs.
3. Most importantly, we defined _inner_decorator that is going to be called each time we call my_function in the code. It accepts positional and keyword arguments and passes them as arguments to the decorated function.

Always keep in mind that the decorator function has to return a function that accepts the same arguments (number and their respective types) and returns the same output (again, number and their respective types). That is, if you want to make the function user not confused and the code reader not trying to figure out what the hell is happening.

For example, say you have two functions that are different in results but also require arguments:

@my_logger
def my_function(n):
    return sum(range(n))

@my_logger
def my_unordinary_function(n, m):
    return sum(range(n)) + m

print(my_function(5))
# my_function is being called!
# 10

print(my_unordinary_function(5, 1))
# my_unordinary_function is being called!
# 11

In our example, the decorator function accepts just the function it decorates. But what if you wanted to pass additional parameters and dynamically change decorator behavior? Say you want to tune the logger decorator’s verbosity. So far, our decorator function has accepted one argument: the function it decorates. However, when the decorator function has its own arguments, these are passed to it first. Then, the decorator function has to return a function that accepts the decorated one. Essentially, things are getting more convoluted. Remember the movie Inception reference?

Here is an example:

from enum import IntEnum, auto
from datetime import datetime
from functools import wraps

class LogVerbosity(IntEnum):
    ZERO = auto()
    LOW = auto()
    MEDIUM = auto()
    HIGH = auto()

def my_logger(verbosity: LogVerbosity):

    def _inner_logger(fun):

        def _inner_decorator(*args, **kwargs):
            if verbosity >= LogVerbosity.LOW:
                print(f'LOG: Verbosity level: {verbosity}')
                print(f'LOG: {fun.__name__} is being called!')
            if verbosity >= LogVerbosity.MEDIUM:
                print(f'LOG: Date and time of call is {datetime.utcnow()}.')
            if verbosity == LogVerbosity.HIGH:
                print(f'LOG: Scope of the caller is {__name__}.')
                print(f'LOG: Arguments are {args}, {kwargs}')

            return fun(*args, **kwargs)

        return _inner_decorator

    return _inner_logger

@my_logger(verbosity=LogVerbosity.LOW)
def my_function(n):
    return sum(range(n))

@my_logger(verbosity=LogVerbosity.HIGH)
def my_unordinary_function(n, m):
    return sum(range(n)) + m

print(my_function(10))
# LOG: Verbosity level: LOW
# LOG: my_function is being called!
# 45

print(my_unordinary_function(5, 1))
# LOG: Verbosity level: HIGH
# LOG: my_unordinary_function is being called!
# LOG: Date and time of call is 2023-07-25 19:09:15.954603.
# LOG: Scope of the caller is __main__.
# LOG: Arguments are (5, 1), {}
# 11 

I won’t go into describing code unrelated to decorator, but I encourage you to look it up and learn. Here we have a decorator that logs function calls with different verbosity. As already described, my_logger decorator now accepts arguments that dynamically change its behavior. After arguments are passed to it, the resulting function that it returns should accept a function to decorate. This is the _inner_logger function. By now, you should understand what the rest of the decorator code is doing.

My first idea for this post was to talk about advanced topics for decorators. However, as you probably know by now, I have mentioned and used a lot of advanced topics as well. In future blog posts, I’ll tackle some of these to a certain extent. Nevertheless, my advice for you is to go and learn about the things mentioned here from other sources as well. I hope I have introduced something new for you and that you are now confident about writing functions as an advanced Python programmer.

References

• Primer on Python decorators
• Python Inner Functions: What Are They Good For?
• Enum HOWTO