Skip to content

Iterating Smarter: Unlocking Go’s New range with Functions

Published on December 25, 2024

Lately, I’ve been exploring some of the newer features in Go, and one that really got me hooked is the ability to use range with functions, introduced in Go 1.23. It’s a feature that, at first glance, might seem like a minor tweak, but once you dig in, it opens up exciting possibilities for writing cleaner, more modular code.

This article is all about diving into this feature hands-on. My goal here isn’t just to explain it but to get my hands dirty with practical examples—and maybe inspire you to try it out too. Let’s figure out together how push and pull iterators can simplify common patterns, make your code more maintainable, and maybe even spark some fresh ideas for your next project. So, grab your favorite editor, and let’s learn something new!

TL;DR

  • Go 1.23 Feature Highlight: Introduced the ability to use range with functions, enabling concise and modular iteration logic.
  • Push Iterators:
    • Centralize iteration logic in the All method, simplifying repetitive traversal.
    • Enable operations like Filter, Search, and Map with minimal code changes.
    • Follow the Open/Closed and DRY principles, making code extensible and maintainable.
  • Pull Iterators:
    • Provide on-demand data retrieval using the PullIterator method.
    • Ideal for complex tasks like playlist comparison or lazy evaluations.
    • Enhance performance and memory usage by processing elements only when required.
    • Allow flexible pausing and resuming of iteration, supporting advanced workflows.
  • Adapters for Enhanced Composition:
    • Use functions like Filter and Map to chain operations on sequences.
    • Promote reusability and clarity in workflows like filtering and transforming playlist data.
  • Performance Insights:
    • Direct map iteration is slightly faster than maps.All, but the latter offers cleaner composition for more complex operations.
    • Trade-offs between readability and performance depend on your application’s needs.
  • Key Takeaways:
    • Combining push and pull iterators with adapters results in highly modular, maintainable code.
    • Iterators open possibilities for handling linked lists, maps, and other structures elegantly.
    • Go’s new iterator features provide both functional programming power and practical use cases for developers.

A Simple Push Iterator Use Case

Imagine that you’re creating an audio streaming service like Spotify or Deezer. You’re requested to develop a feature to display the total length of a playlist. This is the current linked list structure of your company:

Each song has its own duration, but the current song from the playlist points to the next.
Translating this to go code means something like this:

type Song struct {
	Title    string
	Artist   string
	Duration time.Duration
}

type PlaylistSong struct {
	Song *Song
	Next *PlaylistSong
}

type Playlist struct {
	Head *PlaylistSong
}

With this piece of code in your hands you can tell me: “Hey, that’s pretty straightforward. I just have to iterate over each item and get the duration.”.
As a result that’s the code that you produce:

func (p *Playlist) GetDuration() time.Duration {
    var duration time.Duration
    for currentSong := p.Head; currentSong != nil; currentSong = currentSong.Next {
        duration += currentSong.Song.Duration
    }
    return duration
}

The result of this code might be something like this:

Playlist Duration: 16m0s

The result is working as expected and all of your tests are passing flawlessly.
After a few minutes and a few cups of coffee, you receive a code review from your teammate saying:
Hey, I'm implementing a feature to filter songs from a playlist by the artist and song's name. I'm implementing an iterator pattern at PlaylistSong. Why don't you give it a try?

Why can’t she implement it just like me? It would be something like:

func (p *Playlist) Search(term string) *PlaylistSong {
    for currentSong := p.Head; currentSong != nil; currentSong = currentSong.Next {
        // Do search for each element
    }
}

The main problem is this for loop. It is a little messy and you will have to repeat yourself every single time that you want to get your entire playlist. Maybe we shouldn’t have picked a linked list at the beginning to represent a playlist, but it is too late right now for refactoring and you and your teammate have a schedule to follow.
So, you decided to search for the suggested approach and find this article from Ian Lance Taylor telling about this new go feature.
After reading you came up with this solution that fits perfectly for you, your teammate, and any future implementations that might need the full playlist:

func (p *Playlist) All() iter.Seq[PlaylistSong] {
    return func(yield func(PlaylistSong) bool) {
        for currentSong := p.Head; currentSong != nil; currentSong = currentSong.Next {
            if !yield(*currentSong) {
                return
            }
        }
    }
}

func (p *Playlist) GetDuration() time.Duration {
    var duration time.Duration
    for currentSong := range p.All() {
        duration += currentSong.Song.Duration
    }
    return duration
}

func (p *Playlist) Search(term string) *Playlist {
    for currentSong := range p.All() {
        // Do search for each element
    }
}

In Go 1.23, a new language feature allows the range keyword to iterate over functions that return elements sequentially. This enhancement enables more concise and readable code when traversing data structures like linked lists.
To leverage this feature, we define a All method for the Playlist struct. This method returns a function that yields each PlaylistSong in sequence.
Here, iter.Seq[PlaylistSong] represents a function type that accepts a yield function. This yield function processes each PlaylistSong and returns a boolean indicating whether to continue iteration. The All method traverses the linked list, invoking yield for each song until the end of the list or until yield returns false.
Using the All function, we are now able to

use our sequence without having to know about the linked list or any other implementation details.

You are now able to build a function that you have seen in other languages like Filter . With the code below from Taylor’s article:

// Filter returns a new sequence containing elements from 's' that satisfy the predicate 'f'.
func Filter[V any](f func(V) bool, s iter.Seq[V]) iter.Seq[V] {
    // Return an iterator function that takes a 'yield' function as its parameter.
    return func(yield func(V) bool) {
        // Iterate over each element 'v' in the input sequence 's'.
        for v := range s {
            // If the element satisfies the predicate 'f'...
            if f(v) {
                // ...pass it to 'yield'. If 'yield' returns false, stop iteration.
                if !yield(v) {
                    return
                }
            }
        }
    }
}

The only change is that now we have added a f function to check if the element should or not be considered at the final Seq.
And now we are able to implement a filter function into our Playlist like this:

func (p *Playlist) Filter(f func(PlaylistSong) bool) *Playlist {
    result := Filter(f, p.All())
    filteredPlaylist := Playlist{}
    for currentSong := range result {
        filteredPlaylist.AddSong(currentSong.Song)
    }

    return &filteredPlaylist
}

Now, we can simply implement a Search with:

func (p *Playlist) Search(term string) *Playlist {
    return  p.Filter(func(song PlaylistSong) bool { return song.Song.Title == term || song.Song.Artist == term })
}

This way we are allowing the extension of the playlist functionality without modification, adhering to the Open/Closed Principle from SOLID. By implementing the All method and using it as a base for operations like GetDuration, Search, and Filter, we eliminate the repetitive traversal logic that would otherwise appear in each method. This makes the code more maintainable and less prone to errors when changes or new features are required.

Additionally, this approach follows the DRY (Don’t Repeat Yourself) principle by centralizing the iteration logic in the All method. Repeated iteration code across different methods is replaced with reusable sequences, ensuring consistency and reducing redundancy. If the underlying data structure for the playlist changes in the future, we only need to update the All method, and all dependent methods will continue to work seamlessly.

By combining these principles, the code becomes more modular, extensible, and easier to understand. New features, such as sorting or mapping songs to a new structure, can be added by leveraging the same iterator-based approach, further enhancing the system’s flexibility without sacrificing readability or maintainability.

This stupid example shows us how to build a Push Iterator.

Implementing Pull Iterators in Go

Just as you’re celebrating the newfound elegance of the All method and the iterator magic, your product manager comes in with another feature request. This time, you’re asked to compare two playlists and calculate a similarity score. The goal is to measure how similar two playlists are based on their songs and the order in which they appear.

With a push iterator, this would mean writing repetitive, messy traversal logic again. However, with pull iterators, we can make this task cleaner and more modular. Let’s first set up the groundwork for implementing the comparison.

To simplify traversal for comparison, we can use a pull iterator for the Playlist:

// PullIterator provides a pull-style iterator over the playlist songs.
func (p *Playlist) PullIterator() (next func() (PlaylistSong, bool), stop func()) {
    return iter.Pull(p.All())
}

This PullIterator method converts our push-style iterator from the All method into a pull-style iterator, giving us next and stop functions. These functions allow us to fetch items on demand, making it easy to compare two playlists step-by-step.

With the pull iterator in place, implementing a similarity function becomes straightforward. Here’s how we can proceed:

func (p *Playlist) Similarity(other *Playlist) float64 {
    nextA, stopA := p.PullIterator()
    defer stopA()

    nextB, stopB := other.PullIterator()
    defer stopB()

    var matches, total int
    for {
        songA, okA := nextA()
        songB, okB := nextB()

        if !okA || !okB {
            break
        }

        total++

        // Compare songs based on title and artist (ignoring duration for now)
        if songA == songB {
            matches++
        }
    }

    // If both playlists have the same number of songs, calculate similarity based on matches.
    if total == 0 {
        return 0.0
    }

    return float64(matches) / float64(total)
}

But you might be thinking: What are the advantages of using Pull Iterators?

They enable on-demand data retrieval, fetching elements one at a time only when needed, which reduces memory usage and improves performance for large datasets. This approach is particularly useful in scenarios involving comparisons or synchronizations, as it simplifies traversal by allowing step-by-step iteration without preloading sequences. Additionally, pull iterators promote a clean separation of concerns by decoupling the logic for data generation from its consumption, making the code modular and easier to maintain. Their lazy evaluation ensures elements are processed only when required, optimizing resource usage and supporting operations like filtering or mapping incrementally. Lastly, pull iterators offer flexibility in pausing and resuming iteration, making them ideal for complex tasks or intermediate computations without the overhead of maintaining an external state.

A few more cool Iterator features

One of the most powerful aspects of Go’s new iterator capabilities is the use of adapters, which allow you to compose sequences seamlessly. Adapters let you chain operations like filtering, mapping, or sorting without modifying the underlying data structure. Imagine a music app where you want to filter songs by a specific artist, and then transform them to display only the title and duration. With adapters, this becomes straightforward and highly reusable.

First, we will implement a new Map function:

func Map[V any, R any](s iter.Seq[V], transform func(V) R) iter.Seq[R] {
    return func(yield func(R) bool) {
        for v := range s {
            if !yield(transform(v)) {
                return
            }
        }
    }
}

Now, you can use these adapters to create a sequence pipeline:

type SongSummary struct {
    Title    string
    Duration time.Duration
}

func (p *Playlist) SummariesByArtist(artist string) iter.Seq[SongSummary] {
    filtered := Filter(func(song *PlaylistSong) bool {
        return song.Song.Artist == artist
    }, p.All())

    return Map(filtered, func(song *PlaylistSong) SongSummary {
        return SongSummary{
            Title:    song.Song.Title,
            Duration: song.Song.Duration,
        }
    })
}

This setup allows you to filter and transform your playlist data efficiently, promoting code reuse and clarity.

Benchmarking Maps Iteration Approaches in Go

When faced with the task of iterating over a map in Go, a question naturally arises: which approach is more performant—using the classic direct map iteration or the newer maps.All method introduced in the maps package? The maps.All method offers a more concise and expressive way to iterate over key-value pairs, but does this added convenience come at a performance cost?

To answer this, I wrote a benchmark test comparing the two methods. The test iterates over a map containing a few constants and measures the time taken by each approach. The benchmark code looks like this:

func BenchmarkMapsAll(b *testing.B) {
    m := map[string]float64{
        "ε": 8.854187817620389e-12,
        "π":  math.Pi,
        "e":  math.E,
        "ħ":  1.054571817e-34,
    }
    b.ResetTimer()
    for i := 0; i < b.N; i++ {
        for _, kv := range maps.All(m) {
            _ = kv.Key
            _ = kv.Value
        }
    }
}

func BenchmarkDirectMapIteration(b *testing.B) {
    m := map[string]float64{
        "ε": 8.854187817620389e-12,
        "π":  math.Pi,
        "e":  math.E,
        "ħ":  1.054571817e-34,
    }
    b.ResetTimer()
    for i := 0; i < b.N; i++ {
        for k, v := range m {
            _ = k
            _ = v
        }
    }
}

With this setup, I ran the benchmarks to measure the average time per operation (ns/op) for both methods 5 times. Here are the results:

BenchmarkMapsAll-16                     11407765               106.1 ns/op             0 B/op          0 allocs/op
BenchmarkMapsAll-16                     11464362               107.6 ns/op             0 B/op          0 allocs/op
BenchmarkMapsAll-16                     11203790               106.7 ns/op             0 B/op          0 allocs/op
BenchmarkMapsAll-16                     11160464               104.8 ns/op             0 B/op          0 allocs/op
BenchmarkMapsAll-16                     11255577               105.8 ns/op             0 B/op          0 allocs/op

BenchmarkDirectMapIteration-16          11929827               101.9 ns/op             0 B/op          0 allocs/op
BenchmarkDirectMapIteration-16          12066655                99.67 ns/op            0 B/op          0 allocs/op
BenchmarkDirectMapIteration-16          12131480                99.49 ns/op            0 B/op          0 allocs/op
BenchmarkDirectMapIteration-16          11923983                99.21 ns/op            0 B/op          0 allocs/op
BenchmarkDirectMapIteration-16          11976860               101.1 ns/op             0 B/op          0 allocs/op

The average time per operation revealed a slight performance advantage for direct map iteration, with an average of 100.27 ns/op compared to 106.2 ns/op for maps.All. While the difference is small—around 5.93 ns/op—it does highlight that the direct iteration avoids some minor overhead introduced by maps.All.

This exploration sets the stage for a deeper discussion on trade-offs between performance and code readability, which I’ll explore further below.

Disclaimer: The Power of Composition

While these benchmarks focus purely on performance, it’s important to note one of the key strengths of maps.All: the ability to chain operations. For scenarios involving filtering, mapping, or combining transformations, maps.All provides a much cleaner and more functional approach. This flexibility isn’t reflected in the raw performance numbers but can lead to significantly more maintainable and reusable code, especially in complex workflows.

The choice between the two methods should consider not just speed but also how well the method fits into the overall design of your application.

That’s all folks

In conclusion, exploring Go’s new range with functions has been an enlightening journey into how thoughtful features can enhance code readability, modularity, and maintainability. It’s not just about iterators; it’s about rethinking how we approach iteration itself. By integrating principles like DRY and Open/Closed, we can build systems that are easier to extend and maintain, even under tight deadlines. So, whether you’re optimizing playlists or benchmarking maps, Go 1.23 has given us more tools to write cleaner, smarter code—and that’s something worth celebrating. If you haven’t tried it yet, give it a shot. You might just fall in love with Go all over again.

References:

https://go.dev/blog/range-functions
https://bitfieldconsulting.com/posts/iterators
https://en.wikipedia.org/wiki/Don%27t_repeat_yourself
https://en.wikipedia.org/wiki/SOLID

Published inBlog

Be First to Comment

    Leave a Reply

    Your email address will not be published. Required fields are marked *