While working on a data mining assignment over the past two days, I wrote a Python program of significant scale (over two hundred lines) for the first time. It was also my first time truly utilizing Python classes and namedtuple. During the programming process, I developed several reflections on the Python language, which I am documenting here for further exploration when I have more time.
1. Python’s Object-Oriented Mechanism
A very distinct feature of Python class syntax is that the first parameter of every method must be self, similar to the this reference in Java. During development, I noticed that whenever I needed to reference a member or method within a class, I had to access it via self, i.e., self.xxx. I suspect that Python’s class syntax might just be syntactic sugar, where the interpreter converts method calls directly into functions. Since the programmer already includes self as the first parameter, the interpreter doesn’t even need to modify the method’s parameter list. The fact that Python classes do not support access control (public/protected/private) seems to support this observation.
2. List Comprehensions and Generators
List comprehension is a fantastic feature in Python and one of the reasons I enjoy the language. It can implement the functionality of map, filter, and apply from functional programming in a much clearer and more intuitive way. I am very interested in exploring the underlying principles of list comprehensions: are they implemented using for loops? How many temporary variables do they generate? Understanding these details will help me use this feature more effectively.
List comprehensions share a drawback with functional programming: they return a brand-new list, which can significantly degrade performance when dealing with large datasets. Consequently, I am reluctant to use them for massive data operations. However, generators are an excellent solution. They do not generate all results at once; instead, they produce one result at a time when iterated, providing a sense of lazy evaluation. The syntax for a generator is identical to a list comprehension, except that square brackets are replaced with parentheses. When used as a function argument, the parentheses can even be omitted, like so:
sum(i * i for i in range(10))It is worth noting that the range function returns a list in Python 2, whereas it returns a generator in Python 3. If you only perform a single iteration, there is no difference between a generator and a list. However, my impression is that generators can only be iterated once and cannot be reused unless a new generator is created. I once considered using a generator function to pass a generator into a function that originally expected a list, but since the list needed to be accessed twice within that function, I had to abandon the idea.
3. Making Good Use of namedtuple
A language like Python naturally lacks a struct. If you want to represent a graph structure where each node needs to store its content, incoming edges, and outgoing edges, you could use a list with three elements: [item, prev, next]. However, this requires accessing next via index 2, which is not intuitive at all. While programming, I found myself constantly having to remember which index represented which object, which was a major reason for my lower efficiency when using Python.
Is there a way to access it directly using node.next? I felt that using a class would be inappropriate because the syntax is more complex and the overhead would be higher. Later, I discovered namedtuple. I had heard of this data structure before but never understood its utility; now, I finally do.
Node = namedtuple('Node', ['item', 'prev', 'next'])This defines a Node type with three elements named item, prev, and next. When using it, you can access members just like class attributes: node.next. You can also initialize it using the same syntax as creating a class: Node(item, prev, next). However, namedtuple has one issue: like a standard tuple, its members are immutable, which limits its use cases. That said, it seems namedtuple provides methods for update operations, which warrants further investigation.
4. References in Python Lists
Based on my understanding of computer systems and my vague impressions from learning Python previously, all variables in Python should be references pointing to objects in heap space. Therefore, when these variables are passed as arguments, they are passed by reference, meaning the entire list is not copied. Previously, when I wrote code for a Bayesian classifier in Python, I made some variables global to ensure efficiency because I wasn’t clear on the language’s runtime rules. If I had understood that variables are references, I could have written better code. This confirms that understanding computer systems significantly improves programming proficiency.
5. Implementing Linked Lists in Python
Since Python lacks pointers, implementing pointer-based data structures (like trees or graphs) requires certain techniques. I previously saw a cursor-based implementation of linked lists for languages without pointers. However, I didn’t consult specific materials and instead constructed a graph structure based on a dictionary during my programming. A nodes dictionary maintains a mapping from node IDs to node objects. Each node object is a triple (item, prev, next), where item is the node content, prev is a set of IDs for incoming neighbors, and next is a set of IDs for outgoing neighbors. Once an ID is found, the corresponding node can be retrieved from the nodes dictionary. I wonder if there is a better implementation method or if Python has libraries specifically for these kinds of data structures.