The OpenStreetMap.org Codebase, Circa 2024
So, while editing this week’s WN756, I stumbled upon Ilya’s post titled “Сумерки OpenStreetMap.”
It’s a great piece of writing, summarizing the current state of the OpenStreetMap developer ecosystem as of 2024.
Here, I want to translate parts of that to English and provide my own commentary.
Dark Mode
In mid-November, the homepage added a dark mode.
When the browser detects it’s dark outside, it informs the site, which then replaces the white background with black, aiming to spare users’ eyes.
Some people use dark mode day and night (to avoid over-lighting their basement), and for them, this was the most noticeable change in years.
Any changes to the “face of the project,” the openstreetmap.org website, are widely discussed. The dark mode is no exception.
In September, users requested it, but after getting what they wanted, they started looking for ways to undo it.
Shaded tiles are the main issue. Complaints about them appear in a GitHub ticket. Reddit users shared ways to disable them.
Recently, a dedicated toggle was added to the site settings. It’s a rather odd solution but a reasonable compromise from a design perspective.
The traditional approach of inverting colors and rotating the palette 180° works, but it’s not perfect: colors turn darker than necessary, leaving users wanting further adjustments.
The proper solution would be creating a new palette from scratch.
Every shade on the map is the result of a cartographer’s thought process. Colors interact, affect one another, and require contrast, brightness, and accessibility for visually impaired users. Simple inversion ignores all these nuances, resulting in a map that fulfills a practical purpose but aesthetically resembles AI-generated images — unbearable for anyone with artistic sensibilities.
That’s why organizations specifically develop night themes, and this is what was proposed for the osm-carto style.
Unfortunately, there aren’t enough volunteers to maintain two styles instead of one.
The story of the dark mode is interesting for reasons, which we’ll explore in the following sections.
After all, a lot happened last year that escaped the attention of OSM enthusiasts.
One day, the OSM homepage suddenly forced us into using ‘dark mode.’
I hate it, and many others do too. It makes the map dimmer and harder to see.
We protested but didn’t get a solution.
So, we tried to come up with our own. Eventually, someone created a userscript to fix the OSM dark mode issue. I stumbled upon that userscript and decided to port it to Violentmonkey — the userscript platform I prefer, thanks to better-osm-org.
The port worked, and I was happy. Problem solved.
Andy — Anton — Tom
The pull request for the dark mode itself consists of just a single line (plus about ten others to fix minor issues, such as the panel close button). This was made possible thanks to five years of continuous effort by Andy Allan, who migrated the website’s design to Bootstrap. As a result, there was no need to think about colors or adjust components — hundreds of framework developers had already addressed these issues.
The dark mode was implemented and merged by the website’s new maintainer, Anton Khorev from St. Petersburg.
Not much is known about him outside of OSM, but within the project, he is highly active.
Since 2019, he has been a member of the Data Working Group. In 2018, Anton wrote four meticulously detailed articles about mapping establishments, and the other 19 posts in his OSM diary are equally thoughtful and full of practical observations. His note-viewing tool is widely used by those who care about unresolved notes in their city. Anton started submitting pull requests for the website’s code in the summer of 2022 and has remained incredibly prolific, creating about ten pull requests a week.
If you look at the history, there are many drafts, unresolved requests, and debates — but Anton is undeterred and simply continues creating more. One week it’s adjusting fonts, another it’s improving display on phones, and then optimizing code elsewhere. Of the 157 open pull requests at the moment, more than half are his.
It wasn’t until the summer of 2024 that Andy realized it was time to add Anton to the maintainers list.
So, while it might seem that the OpenStreetMap website hasn’t changed for years, there is constant activity behind the scenes — things are being reorganized, cleaned up, or made more convenient. In October, Andy transcribed his talk from the London conference about his experiences working on the website, offering insights into issues like translation challenges and securely rendering HTML.
However, the impression that “nothing is happening” isn’t entirely unfounded. Whenever someone (even a maintainer) tries to make a major change, Tom, the long-standing site steward since 2007, often refuses to engage and stretches discussions indefinitely.
Back in the days of maps.me, I made several pull requests, but they were all closed for various reasons. I gave up on the site, and dozens of others left for similar reasons. Those who remain focus on polishing code and tweaking HTML formatting instead of modernizing the site or addressing questions about “what the site says about the community.” If refactoring takes five years with nothing to show for it — what’s the point?
At one point, I also interacted with Andy, Anton, and Tom.
Back then, I tried to port a certain functionality from better-osm.org (better diff visualization for changesets) directly into the OpenStreetMap.org codebase. However, I gave up halfway — it was just too hard.
At least for now, I’ve solved my own personal problem: being able to see better diff visualizations for changesets. Why should I bother gifting this wonderful (?) feature to everyone else? I’d like to help, but my own capabilities have limits. So, I gave up.
‘Back in the days of maps.me, I made several pull requests, but they were all closed for various reasons. I gave up on the site, and dozens of others left for similar reasons.’
Yeah, me too. I also gave up on the site. Haha. But in my case, I closed my own pull request.
‘Whenever someone (even a maintainer) tries to make a major change, Tom, the long-standing site steward since 2007, often refuses to engage and stretches discussions indefinitely.’
This brings us to the age-old debate: change vs. stability.
Major changes can make things worse. For example, this dark mode is quite a significant change, and it’s clearly made things worse. But without change, things can’t improve either.
On the other hand, stability is valuable. However, if taken to the extreme, constant stability stifles growth.
My point is that we probably need to strike the right balance between ‘changing things up’ and ‘keeping everything stable.’
Contributing outside the OSM codebase (1)
Where do developers go?
Let’s return to the dark mode.
Before November, the site was darkened using browser extensions. User scripts and browser extensions, which don’t require website authors’ approval, are an excellent way to quickly add small features without drawn-out discussions that risk going nowhere.
The most impressive extension of the past year comes from Roman Deev. His better-osm-org adds an “Edit with Rapid” option to the editing menu. It also makes lists more compact, displays geometry on changeset pages, highlights tag changes in history, and even integrates likes from OSMCha.
Could these minor features have been integrated directly into the site’s code?
Absolutely
But this would require not only negotiation (or submitting daily pull requests for two years to earn trust) but also learning the project’s Ruby on Rails architecture.
Eighteen years ago, this framework was new and trendy, but today, developers familiar with it are hard to find. That’s why people prefer more widely-used languages like JavaScript (which powers browser extensions) or Python.
Better-OSM-org is the GOAT. I use it on a regular basis.
Contributing outside the OSM codebase (2)
Kamil Monich (better known as NorthCrab) has been rewriting the entire OpenStreetMap backend in Python, almost single-handedly, for over a year now.
The OpenStreetMap NextGen project is intended to replace the current massive codebase: Rails Port, CGIMap, and the scattered import-export scripts. If successful, contributing to development will no longer require knowledge of at least three outdated languages. Instead, it will rely on two of the world’s most popular technologies: Python, using familiar libraries like FastAPI and SQLAlchemy, and JavaScript.
How can this seemingly simple tech stack compete with 18 years of refined code, let alone with C++? Kamil counters that most of the time, the code is simply waiting for PostgreSQL to respond. As Paul Norman pointed out, Rails Port could have worked without C++, but the Ruby queries have become outdated because they were neglected in favor of CGIMap.
One year and three months after its announcement, development shows no signs of slowing down. Every month, Kamil showcases new engine features, which at times resemble API 0.7 or Roman’s better-osm-org extension. The roadmap has been rewritten several times, and the demo server’s launch date has been postponed repeatedly — from February to May, and then further and further.
I know about this project, but I’m watching it cautiously because I still have some skepticism about it.
First, I wonder whether Python could outperform the currently C++-optimized OSM in terms of speed and raw performance.
But, I’m not sure if OSM’s C++ code actually contributes to the performance. I wonder if that C++ codebase is just a workaround to solve the Ruby query bottleneck. Could calling PostgreSQL directly from Python outperform the C++ implementation?
The second concern is that, up until now, I still can’t access the public demo.
Sure, he’s posted repeatedly about the project’s progress, but there’s still no public demo that anyone can access.
“Additionally, the demo server’s launch date has been postponed repeatedly — from February to May, and then further and further.”
This adds to my concerns…
The Funding
On January 1st, the NLNet Foundation unexpectedly announced 50 recipients of NGI Zero grants, and among them was OpenStreetMap-NG. This means that within six months to a year, OSM-NG will load all 14 terabytes of the database (most of which will be compressed into a single table). Then we’ll find out if all these promised speed improvements are real.
At that point, we’ll face the question: should Tom, the long-time gatekeeper of OSM, be replaced by Kamil? And was Anton’s marathon effort of 700 pull requests for Rails Port worth it?
But it’s worth noting that OSM-NG wasn’t the only grant recipient last month. The OSM Foundation itself received €178,000 from Meta (equivalent to 2% of their annual membership in Overture Maps) and, the next day, an additional €384,000 over two years from Germany’s Sovereign Tech Fund.
Of course, there won’t be any surprise splurges with these funds. The usual allocations apply: conferences and infrastructure. However, the latter was outlined in slightly more detail this time: “modernizing code to current standards, attracting developers with better documentation and tests, and researching topics like vandalism and new ways of working with data.”
The OSMF Board correctly recognized that things won’t get done on their own — a dedicated, paid individual is needed to guide and assist development. Last week, the OSM Foundation published its first job vacancy since 2021. Until January 21st, they’re seeking a Development Coordinator for the OSM website and related projects. The ideal candidate will have project management experience, familiarity with open-source contributions, and possibly even prior involvement with the OSM community. Ideally.
Both OSM-NG and OSMF have received funding to make OSM even better. Great. What a competition.
At this point, developers from the OSM community can choose which side to support. Which project should they invest in and help with? The original OSM from OSMF, or the newcomer OSM-NG? Cool, I guess.
For now, OSMF has already made its first move: posting a hiring announcement for a position to help with the project.
Cool.
By tugging on the thread of a seemingly minor change on osm.org, we’ve unraveled a story that casts 2024 in OpenStreetMap in an intriguing light.
This was the year when dissatisfaction with the leadership of OSM’s central site sparked multiple independent initiatives to improve it, bypassing the usual pull request process.
Roman’s browser extension and Kamil’s alternative site not only challenge the notion of osm.org as an antiquated relic but also raise a provocative question: if the gatekeepers weren’t so strict, could we have enjoyed new site features and API improvements every month for all these years? Is the stability maintained by entrenched, immovable “elders” of OSM development truly worth it?
“Is the stability maintained by the entrenched, immovable “elders” of OSM development truly worth it?”
Well, personally, I would say yes to this. Haha.
If someone is really dissatisfied with the current OSM codebase, they could simply rewrite it from scratch, just like Kamil did. Everything is open, and no one is stopping you from doing so. This is why right now we have a huge variety of OSM editors, which come in all shapes and sizes.
This is just my personal opinion, though. Feel free to disagree. Haha.
