Dammit! I am a software engineer, right? This stuff is so complicated! Wow!

The more you understand, the better. You can do more. We are all trying to get up to speed with more technologies. There are levels of expertise.

The first is the ability to actually use the tool. I mean the ability to write business logic. The ability to have some folklore idea of “I think things should happen this way” and to actually implement it. A lot of these ideas for people like me are workflow ideas. “I currently cannot get project-wide diagnostics in lsp-mode when I open only one file. I need to open all files in order to get diagnostics form all files. Let me write a package that auto-opens all files under the project root.” This is a tier of knowledge. Getting here is hard. You need to understand what session, workspace, client, etc mean in the context of lsp-mode. It is not the easiest.

This maybe seems like essentially knowledge of the API, but what it really comes down to, and what you will need to know in order to have confidence in what you are doing, is knowledge of the control flow. So you need to read source. Bahgawd that is really hard. I mean it is its own kind of difficulty.

Okay, and then there is another level of knowledge at which you can start deeply innovating in this “deeper tech” itself. Like making improvements at the actual LSP “major implementation” or even standards level. So here you have people like yyoncho who actually write things like lsp-mode. I am not here so I cannot talk about it.

The first-level difficulty is like that of python tooling and infrastructure. Any 12 year old can learn python “if-then” business logic. But it is not easy to understand when you would need to use a manually created venv in the dev process. Should every python project have its own lsp venv containing pylsp and other packages? I shouldn’t really use the global python system so… etc. Or choosing a “project management tool” like Hatch. And compare it to older things like setuptools. To understand packaging and doing it correctly. To know what Poetry and Tox are and whether to use them. Pipx and Pyenv. Modules and imports. Static analysis, mypy, ruff, pylint, testing. “Contravariant generic types.” This stuff is the level 1 python knowledge. The mere ability to write business logic “if-then” in an online python editor is level 0.

It is tough to get to this level 1. Thank god for open source, so you can read the code, cause lots of docs… could use work. All these programs were written by other devs with their own worldviews. I mean logical worldviews. There are certain design standards, sure, but I think we are all so different with regard to how we think, at a deep, mechanical level. This comes across when you read other people’s code. Almost all the code I write, I feel is very intuitive. I mean: when I structure my project in some way, the components, the control flow, I am thinking: this is what the universe wants for my project. This feels “objectively correct” to me. It is my intuition. It is natural. It is clean.

Is it a question of experience and coding style? Is there one actual objectively correct design pattern which we will all converge to with sufficient thought?

Then I read some other person’s package and it is gibberish. Not the code. (Well sometimes the language itself, if I am not sufficiently familiar.) But the layout, the control flow. Getting to that level 1 at which the “head has been wrapped” and I can start putting my own business logic on top of their package.

Docs suffer from the same issue. Because at the end of the day, even an english-language explanation is subject to the same influences of the intuitive logical worldview of the person who wrote it. And they perhaps will be different from mine.

After I understand someone else’s project, I have a mental model that makes sense to me. And I also have a bunch of ideas along the lines of how I would make the project if I were writing it. I would call this object something different, or the “entry point” would be here rather than there. Or this function would be a method of this class instead of that.

All I am saying is that this is a challenge. And I am not sure it should be so much of a challenge. I think everyone experiences this. Perhaps we do not talk about it because we are afraid to look dumb. I know that is why I have rarely spoken about it myself. I think this problem of the general difficulty of software engineering, this problem at least of understanding and getting to level 1, is something that can be solved. Or at least it would be real cool. I am not talking about “solving” it by the nebulous heuristics and experience that one accumulates by time and practice, and getting better that way. I mean actually by standards or something. Maybe not. Just looking for a more intentional way to go about this.