Avoiding pitfalls when implementing a partially-undefined areas of the design document

Haha, I think I am worse than you, I mostly jump around between different projects, and of course, as I have no system in place to track progress, I sort of forget what I was doing the last time, and somewhat start again with the same project when I return. :D

Not that I mind though, I write big programs at work, I don't want to do that in my spare time as well. One job is enough. I use my spare time for writing small or interesting programs. I do spend time thinking why it happens and how I could change that behavior though.

As for the topic of avoiding pitfalls, I found several ways to avoid getting lost in the maze of arbitrary coding. For difficult problems I do a lot of thinking how to implement things. When I have a vision of how it should be done, I can code that without derailing into other subjects.

For simpler problems, focus on the end-result also tends to work for me, even when the implementation is not fully clear. You need some kind of global path what to implement in which order though, but that is not too hard. Distractions from the goal then end up as comment “// XXX Enhance this by implementing blah blah bla” for future decision (most likely never, but writing that line gets it out of my head).

For an editor experiment I used a feature list like below. Just a text file with lines and a “[ ]” prefix in front. Near the bottom you add new features to do, you shuffle features roughly in implementation order, and when done, change “[ ]” to “[x]” .

[x] (v1) Display some static text with a cursor.
[x] Move cursor vertically
[x] (v3) Move cursor horizontally

// Horizontally, cursor jumps due to different character widths.

[x] Replace list of lines with gapbuffer.
[x] Add 'change' concept
[x] Add edit stack
[x] Add letter
[x] Remove letter
[x] Join lines with delete/backspace
[x] enter key
    (v4) cursor arrow movement, insert letter, enter, backspace, delete
[x] position cursor with mouse
[ ] Implement jumpy movement (DocumentView.moveCursorHorizontal function)
    Use tokens for jumping.
[x] block selections with ctrl+cursor
[ ] block selections with double click
[ ] block selections with drag
[ ] block paste https://docs.oracle.com/javafx/2/api/javafx/scene/input/Clipboard.html
[ ] block delete https://docs.oracle.com/javafx/2/api/javafx/scene/input/Clipboard.html

[x] Add tokenizing of line
[x] Add notion of language
[x] Add scanner support

[ ] loading files with tab
[ ] indent-size setting
[ ] tab-size setting
[ ] tab
[ ] indent (tab)
[ ] dedent (shift-tab)
[ ] auto-indent

It did work quite nicely, especially when the features are small, so you could perhaps break a big feature into several sub-features. Didn't try that though. I did do some thinking beforehand about main data structures though.

[Off-topic, but while building that editor, I aimed for a simple way to implement the features, that is, nothing smart, nothing special case. What amazed me is that it started to behave like much of the editors we use daily. One of the result for me was thus that we have a world to gain on editor implementation.]

Your post did make me think why the above things work, and it made me remind the book “Freedom From Stress - How to Take Control of Your Life” by David Gamow. One of the things he explains is why we can give advice to others but not to ourselves. The reason for that is being emotionally involved according to the book. You need some distance from the subject to make proper decisions about it. Perhaps this “focus” thing of me is one way to create such distance. Likely there are many more ways to create it.

@alberth Thanks for the input. We have done very similar things, it appears. Don't be too sure that you're worse! I do the little projects all around too. At least I manage to finish those.
The todo thing I also have, but it has grown too much. I feel li

Alberth said:

For difficult problems I do a lot of thinking how to implement things. When I have a vision of how it should be done, I can code that without derailing into other subjects.

I do too, but I'm sure it's because I haven't been specific enough in the “how to implement” things. The steps tend to grow in size, and while the overall/global plan is still ok, I think it helps to have a lower pain/time threshold.

Alberth said:

Distractions from the goal then end up as comment “// XXX Enhance this by implementing blah blah bla” for future decision (most likely never, but writing that line gets it out of my head).

This also makes perfect sense, and is something I will try to do more now.

Alberth said:

[Off-topic, but while building that editor, I aimed for a simple way to implement the features, that is, nothing smart, nothing special case. What amazed me is that it started to behave like much of the editors we use daily. One of the result for me was thus that we have a world to gain on editor implementation.]

That must be very satisfying. Often I've made a prototype, or some very useful, yet quite simple or manageable code base, and thought: Oh, that's all there is to it. Well we can improve on this and actually compete with some of the best [XYZ products].
I haven't gotten much further, but reaching that functional prototype is very satisfying, I think.

Alberth said:
One of the things he explains is why we can give advice to others but not to ourselves. The reason for that is being emotionally involved according to the book. You need some distance from the subject to make proper decisions about it. Perhaps this “focus” thing of me is one way to create such distance. Likely there are many more ways to create it.

That makes sense. It's a cousin to “Kill your darlings”, and I knew I had to kill “project arc edge” in my graph, to proceed. Who knows, maybe I'll get it in eventually. I agree that it's important to get that distance, to be able to cut through with decisions like that. I still have ways to go…

SuperVGA said:
I do too, but I'm sure it's because I haven't been specific enough in the “how to implement” things. The steps tend to grow in size, and while the overall/global plan is still ok, I think it helps to have a lower pain/time threshold.

I keep things quite small by figuring out the fundamentally correct way of handling the cases in the problem only (with “difficult" meaning algorithmically difficult mostly). This is likely like your global plan. Implementation details of a part of the plan then follow from the input and required output of a part to make it fit in the high level solution. Code is usually not big, but doing the right steps at the right point is what needs to be solved.

SuperVGA said:
That must be very satisfying.

It was quite disappointing to me in fact. I am used to vi, so I already considered the behavior of some “normal” editors quite stupid (eg join 2 indented lines and get the leading indent of the 2nd line as whitespace in the middle of the line above!). I expected that my simplistic implementation would be even worse, but it wasn't.

A fun thing I found was why many normal editors have an empty last line. The root cause turns out to be the difference between seeing a newline as line-separator (normal editors) or line-terminator (vi).

SuperVGA said:
I agree that it's important to get that distance

This discussion is a step. I currently also think that an explicit separation between thinking *about* the project (and its progress and its important steps) eg at fixed points in time, rather than always thinking *in* the project (new features, new code, new details to do) could go quite far. I think *about* the project should be less than a page, but not clear what one should write there.

The thing that keeps me as "on track" as I manage to be is that I write down my thoughts as I'm coding. For a particular coding session, I look at my notes from the day before, write a short bit of thoughts about what direction I'm planning on going and then as I'm working write down whatever problems I might encounter or justifications I have for going with a particular solution. In code, I may leave a comment with just the date and I know that I can use that whenever I need to figure out what the hell I was thinking when I originally coded something. I have a long todo list, a long list of outstanding bugs I'll probably never try to address, and a list of things that are actually done. It doesn't really stop me from going off on some tangent, it just mostly helps me remember what I was doing last session days or months later if need be.

I've been working on the same endless RPG project for what looks like 9 years this coming September which includes code that I originally wrote probably 10 years before. I work on this project more because it's fun and feels really good to be have no constraints on what I'm coding but my own. I don't have a deadline, I don't have coding standards I need to follow (just what seems sensible enough to me), and I don't need to justify working on a feature that I'd like to work on.

And with that all said, the last tangent I went off on was some climate and biome work for world generation. I really wanted to get this to work but it wasn't too long before I realized it was going to take awhile and I promised my kids that I'd get something together so that we could work on making a basic RPG kind of story thing together. So yeah... lately I've been back to working on the UI system to be able to implement some kind of combat that I promised them isn't too far off from being done. Summer's almost over and I think I just managed to untangled the mess of code I needed to be able to expand the UI stuff I needed to. A couple bugs to fix but hopefully I can put together some kind of combat that is up to the standards of my kids.

I'm not so sure it's possible to really escape the usual pitfalls.

Alberth said:
I think *about* the project should be less than a page, but not clear what one should write there.

Who is on the project. More people needed? Somebody not pulling their weight compared to the others?

Any tech you'll be needing that you don't have, or is old tech wearing out? Licenses expiring? Anything that will enhance productivity?

What is the overall goal you hope to achieve by making the game, and what are the remaining steps needed to achieve that? Which step is most crucial to success?

If there are multiple areas needing your attention, rethink your priorities from the previous sprint. Which thing needs to be done first, how much of that can be done in the next sprint.

If a release date is not planned, consider how long a dev cycle would be too long and how soon in the future is surely too soon to happen.

Consider the platform owner. For example, if you're making an Android game, consider the submission process and costs for the Google Play store, or whatever it might be called today. Sure, you've thought about it before, got a plan, but just open your mind to it for a moment, and think again about first steps, preparation for those steps, how far in the future is that.

If you are not only the producer but also the marketing person, think again about the marketing plan. Anything you need to do to keep the campaign moving at an appropriate pace building to release?

Always keep in mind the end user. All UI and feature decisions need to take the users into account.

It's a “forest for the trees” kind of thing, as discussed previously. At the end of every sprint, step back and take a long look ahead as best you can, to where it all comes together. Maybe most of the trees are already behind you. Or maybe you've only scratched the first tree, with many more to go.

I don't think that'll fit on a page. ?

SuperVGA said:

Do you (hobbyist) guys do thorough implementation documents, feature breakdowns etc? I suppose it is actually very good to have for a large project, but I always thought of it as a lot of overhead - not getting down to business and instead writing about stuff that's inevitably going to change.

I think that type of non-plan approach only works well for extremely small projects.

If you can hold the whole thing comfortably in your head then you don't need plans, whether those are blueprints or diagrams or charts or maps or whatever.

When a project begins to be big enough that you can't hold the whole thing at once, whether that is written blueprints or system diagrams charts, it doesn't matter if you're building a shed in the yard or a software program, once it is big enough you need to start writing specs that are detailed enough to work out the problems before you start.

Tom Sloper said:
I don't think that'll fit on a page. ?

Just write very very tiny ?

I fully agree with you though, for something (semi-)commercially viable that you want to publish / sell to an audience indeed you need to consider many many things.

What I was trying to achieve in one page was only about some basic form technical management (as in, steering code development direction somewhat), the topic of the OP. How to avoid derailing into some niche area coding a feature with no or little meaning to the project as a whole.

I think currently that having a better separation between coding (where your head is fully buried in all the little details of the problem) and managing (where you look at the global state of the project, and the best next area to work on) should go a long way in the right direction.

Alright guys, many thanks for all the input. It's super-cool to get feedback from others who have experience with this.

I also work on large projects on my dayjob, and after a re-scan of my little pet project, I had confirmed the suspicion that is is in fact also a large scale project.

One could go all-in with a gold-plated issue tracker, but the advice I've collected applies to people who use notepads, post-its and todo-files.

It's very likely that it's not quite enough, and on should be even more organised when developing on a large project.

Like @frob said; a non-plan approach only works well for extremely small project: Even small projects are easier to implement, understand (and resume) when a plan is available.

Regardless the items that I've collected from your input may still help with staying focused/organised:

• What steps are the most crucial to get a working product? Keep those in mind and prioritise accordingly (consider the end user - their experience should match the requirements).
• Consider working in sprints to better dedicate some time to certain tasks. Consider moving outstanding tasks over or into the backlog. Look at the roadmap after finishing a sprint, to see if things are coming together as fast as you've planned.
• Get "not really important" tasks into todo comments, into a (nested and prioritised?) todo list or create a work item on it, to get it off your mind and stay focused.
• Consider adding a date to todo comments. It's an easy way to mark stuff, so that when it obsoletes or fades into obscurity, you can likely just clear that comment.
• Take notes while developing. Doesn't need to be organised, but needs to be understandable by future you, so you can consider it (a bit) later. Look at notes from last session when starting, and decide what tasks you need to do first this sitting.
• Listen to others, perhaps even listen to yourself (speak out loud, talk to rubber duck). Invested discussions are beneficial!
• Get some distance to project. Either through a perspective provided by others, or by being the PO who like to see stuff done.
• Kill your darlings. The stuff you enjoy doing most (or spend most time on) might be better off simplified or done with a different approach.
• Make sure there is an implementation plan for every task in the overall design/task: This needs to cover the overall approach, and "solve" the problem in a particular order. This is so that you don't stray too much, and have to think things through before jumping in. It's important to keep this separate (but associated) from the overall plan, as to not clutter it. The implementation plan is what happens *in* the project, while the overall plan is *about* the project.

@Tom, some of your points look like they belong on a different level. From there, one would also be able to tell what corners to cut, what things to not do right now etc. But I don't feel like they apply to making the programmer more organised and less likely to enter pitfalls directly; HR, Marketing, Distribution, Licenses and the tech stack.