Tuesday, January 7, 2025

Feature vs Goods, or Estimations vs Real effort, and the iterative nuances of software development

Here we have the classic forms of common sentences circulating in the corridors of (almost) every single team on the world:
 

- "When will feature Y be ready?"'

 - "When will this feature X be delivered?"

- "How much effort will feature Z require?"

One would suspect, there is nothing abnormal to (effectively) "plan ahead", as such, and wanting to know the future. Well, to my experience, these questions have at least one (inconclusive) valid answer: "When it's ready".  And this answer defies the question, of course :).

In my experience, also, providing good estimations (note: _good_ estimations, not *exact* estimations) is a hard hard-problem (yes its twice as hard). There is in fact two effort values at play - first, a tentative / speculative one; issued _before_ the implementation; and second, the *real time spent*, which accrues through the time spent for realizing / implementing a given feature X. 

And Estimating is a science of it's own, really. In fact, there is different types of estimations, both in waterfall and in agile / scrum (hard estimations), or only in agile (guesstimations).  A guesstimate is what it means, a guess at best. This usually is the _first_ value you get out of a developer that just got caught in the "How long can this feature take to implement?" type of trap question. Given its provided out of the blue, by a developer who is - most likely - focusing on something else; and knows most likely zero of the details (which are all required to get a true idea of how much it might actually take) it is a guess at best. 

How are estimations born? If you ask a carpenter:

"How long will it take you to build me a house? " he will ask : "Well what kind of a house?". So it seems that in order to get more precise estimations, we need a certain level of detail. This is "easy" in the non-software world, in a way that things are standardized. Materials are known, tools are known and usually simple to use; processes are clear and straightforward; tasks are solid a well-defined so they can be easily allocated. The carpenter will even know how many bricks will be required to build the house you want. Simple math. Done. But what are the bricks in software development?

In software, things are not as straightforward, mostly due to the nature of it - software is abstract, and more importantly, software development is an *iterative* process. Even if you need to "just build another house (software)"; the  house (software) that you will build is (usally) a totally different beast from the last one; starting with the materials, which migh be something unkown and new; and the tools that are  required to be used to implement it. Also, software is not concrete in a sense like a house is - as in made with bricks, that you can actually touch and carry around, and count.
For instance, any team member will have to shape it's own understanding of a given feature. N team members? N perspectives on the specification of feature X; N _versions_ of feature X.... Communication can mitigate this by having everyone on "the same page", but it's challening, especially when you are in time constraint (which usually is always the case).  How many features will you need in order to deliver the user stories in question?

So if it's challenging to agree on what feature X actually really is, within the team, how can we really even think about agreeing on a given effort value for feature X?

Let's consider waterfall and agile approaches.
Agile solves this with the help of User Stories and slicing into smaller items. Waterfall, with relentless ironing out every. single. detail. of the feature itself, the _implementation of feature_ and (lastly as it's waterfall) the *testing* of the feature. Well then planning helps, no? As we have seen, it does perhaps help in delivering (i.e. having something to ship); but it fails at making sure that what you deliver is actually what can be used by the rest of the world, or, by the users.

Waterfall comes from the old factory lines. And as such is suited for a production line, where the outcome is a good, not a product. We create _the same_  good, X amount of times, _in the same way_ (which btw must be the most efficient in terms of cost and time). Let's take cars for example; every car _has_ to be the same. Same features: same number of windows, same number of wheels (thank you!), same number of pistons in the engine; and so on.

But features are _not_ the same as cars. Every. Feature. Is. Different. Let me write this again, in caps: *every feature is different _and unique_* !!!  Thinking that one can find _one_ process to implement _any_ feature in _the same_ way, is simply limiting,  counterproductive and silly. Features are _not_ goods. An individual feature is more like a *model* of a car, which as such requires a specific tailored *assembling process*. You see where this is going, right? And we haven't even started talking about the fact that the users might not really *want* to drive the models of cars the factory produces.

So as you can see, the space of possible outcomes to implement feature X explodes in the software world even before we start implementing it (and we have also not talked about the variety of tools / technologies etc yet!), which makes effort estimation even more challening.

Now how would a programmer know how much time it takes to implement feature X? Yes, there might be a spec, and predefined tools to be used ( programming language, development environment, testing environment etc). But that's merely the starting point, as we now enter the _implementation process_. 

A common implementation process can be described with a list of "activities" to be done (in agile, referred to as "Definition of Done"). It can look like this:

- The software has a clear and well-understood specification

- The software is implemented

- Relevant unit-tests are implemented as well, with a given coverage (say 80% if you like the number), and no unit-test fails, ever.

- The code for the given feature X is reviewed and merged into a main repository

-  QA is run on the defined user stories.

The question here is now, can this list of activities help us achieve a better estimation?

In fact, it provides some detail, at a very generic level, which can be applied to any feature that we might want to implement; and as such, segments the effort nicely towards a - potentially realistic - value of an estimation: as each developer can simply tell how much time it will take her to understand the spec, implement the feature, write the tests cases etc. This is the clear, straightforward side of the estimation process: based on a concrete list of facts (the "bricks" for instance).

But of course there is also some nuances here. These are a bit more subtle, as they show only deep within the process of software development, and are of _iterative_ nature, unfortunately.
Four relevant insights can be clarified in question form:

- How much effort will it take for a developer to prepare the implementation of feature X?

- How much effort will it take to get all the tests to _pass_? 

- How much effort will each QA round take?

- How many rounds of QA will be required *before* the feature can finally be deployed?


These are the big unknowns that put the first "hard" in the "hard hard-problem" part, and these questions are the _bricks_ of the software development (house).
When you find answers to these, you answe the question "how many bricks do I need to build the (software) house?"


Friday, March 8, 2024

#chatgpt - an adventure; with codereview!

 So there's alot of fuss around AI lately; it's been stirring my curiosity for a while - it's been a long time since playing with it in chess games at university.

For fun, and to explore; i deciced to want to make a "maze" game. A very simple game in which you have to figure out how to get out of a maze, while avoiding enemies.

Nothing fancy, very basic prototype level stuff. Only one requirement: let chatgpt do all the work!


So far, I have:

- a maze

- a player

- two enemies

- a rendering loop

- collision engine

- and some stats

The first funny thing; it calls python "pseudo-code" (i.e. it coded it in python from the beginning). There was a bit a hassle with setting up the dependencies for the OpenGL piping (on Debian), but otherwise it runs. I had a few iterations on it; code restructuring was particularly painful (apparently class structure context is something that gets easily lost when extrapolating from a single file to individual ones - i.e. one per class); when I decided to move the maze code into it's own class, the references to it woule have been still "maze", but it didnt acknowledge the actual maze data present within the maze class (as in maze.maze was missing from the other classes using it).
Most of the logic is then also glued into the rendering class, for some reason - I wonder if  the notion of the inversion of control pattern is tricky for it.

So after this output, we get this:

 

quite an open maze, I must say.  I will try to solve this problem later on.

The stats also do not show, despite being there (there should be a header with a life counter).

The whole process of prompt "engineering" (or whatever it is heretically called) is just brain dumb. Telling chatgpt what is needed is actually _very_ tedious, and can easily lead to errors. Another thing : chatgpt _cannot_ commit to github. Now that's quite the deal; yes of course chatgpt is storing every change for you, and it can revert to any of them at any point; but if you'd want the flexibility of git in it, it'all download/add/commit/push/merge cycles. Unhandy to say the least.
Also, when you ask it for a change, chatgpt will stay specific, i.e. it will prefer to output a short part of the code you are asking changes for, instead (for example) of the whole file with the new changes in it (this makes copy pasting tedious too). It's also "missing out" on some module imports for constants (mainly DOUBLEBUF, OPENGL, which come from the pygame module). As well the font initialization part was missing.

Now lets say I want to add some extra logic (If I click on a wall, I want to be able to "move" the brick by one cell forward or backwards). This is the changeset chatgpt proposes:

48a49,50
>         self.dragging_wall = False
>         self.dragging_wall_pos = None
72a75,78
>     def handle_mouse_down(self, x, y):
>         x = max(0, min(x // 20, self.maze_width - 1))
>         y = max(0, min((y - 50) // 20, self.maze_height - 1))
>         if self.maze.maze[y][x] == 1:
>             self.dragging_wall = True
>             self.dragging_wall_pos = (x, y)
>
82a89,97
>     def handle_mouse_up(self):
>         self.dragging_wall = False
>         self.dragging_wall_pos = None
>
87,89d101
<         # Handle dragging wall
<         if self.dragging_wall and self.dragging_wall_pos is not None:
<             x, y = pygame.mouse.get_pos()
94,107d105
<                 x = max(0, min(x // cell_size, self.maze_width - 1))
<                 y = max(0, min((y - header_height) // cell_size, self.maze_height - 1))
<                 if self.maze.maze[y][x] == 0:
<                     self.maze.maze[self.dragging_wall_pos[1]][self.dragging_wall_pos[0]] = 0
<                     self.maze.maze[y][x] = 1
<                     self.dragging_wall_pos = (x, y)
<
95a108
>     def draw(self):
102a116,119
>             elif event.type == pygame.MOUSEBUTTONDOWN and event.button == 1:
>                 self.handle_mouse_down(*event.pos)
>             elif event.type == pygame.MOUSEBUTTONUP and event.button == 1:
>                 self.handle_mouse_up()
108,109c125,126
<                 glVertex2f(x * cell_size, y * cell_size + header_height)
<                 glVertex2f((x + 1) * cell_size, y * cell_size + header_height)
---
>                 glVertex2f(x * cell_size, y * cell_size + header_height)
>                 glVertex2f((x + 1) * cell_size, y * cell_size + header_height)
111,112c128,129
<                 glVertex2f((x + 1) * cell_size, (y + 1) * cell_size + header_height)
<                 glVertex2f(x * cell_size, (y + 1) * cell_size + header_height)
---
>                 glVertex2f((x + 1) * cell_size, (y + 1) * cell_size + header_height)
>                 glVertex2f(x * cell_size, (y + 1) * cell_size + header_height)

But hey it's been one hour tops, and I am this far! The enemies move, adn the player is controllable. Impressive. The same game took me about 6 hours to code a few years back (mainly the maze algorithm was a bit cumbersome).

 

Sunday, February 25, 2024

Jira Herp derp #2

 So Jira.

At it again.

Damn.

It used to be a decent tool (avoiding flame wars here /flame:off).

But recently with all the cloud jazz from atlassian, life has become impossible with jira.

Sorry jira, our relationship is not working anymore. It's not you (but it is), it's just that we want...different things, I guess.

You want to make more money for yourself, to be able to sustain your growth (and buy shiny new features, which will clog your already clogged UI).

I want to have a simple UI, which I can use to track my acitivity and integrate with my CI cycles.

You want to make sure everthing is secured properly; and because of that, you go and hide in the cloud, as if that would be a safe place to be. And not only that, you take all your companion apps with you!

I want to be able to install/manage relevant versions of you according to my needs, be it with the basic jira or with the companion apps. After all, I have paid for your already (see previous post).

Why should I have to pay (alot) more for :

- the same functionality

- in a foreign environment (the cloud)

 just because <insert reasons here atlassian>?


Disagreements, by grades of severity

During my work life I have come to the conclusion that in fact, all that matters are interactions with people; meaning the people you work with, as well as the interactions one has with the others in the context of a team and/or group; specifically a software engineering/developing group in this (and my) case / experience.

It is relevant that the quality of the outcome is affected by the quality of the interactions; and least not only that, also one's experience on the team will be affected. So communication is affecting (of course) a big part of that outcome. Given that communication, then, plays such a big role in the quality of the outcome, one has to realize that conflict, in a team, plays the opposite role, i.e. degrades the quality of the output. Conflict is one of the team traits that I feel are often mentioned; but at the same time its expected from everyone to be capable of handling it. Personally I don't like the word "conflict" itself. Its too broad of a term; when we talk about "conflict", it's very uncertain what we mean. Armed conflict? How would that translate into a Software development team - armed assault? Verbal violence? Personal attacks? Or fad arguments against one's own opinion? Passive-aggressive comments? Banter chit-chat in the interested person's absence?

Conflict is a dangerous word. Back in the 1920's, futurists (and later fascists) belived the world could be changed (for the better) only with conflict.

Fast forward 100 years later, the world has suffered the biggest armed conflict in human history, with millions of lives spent; the Futurists (which by then had become Fascists) had proven the way of changing the world they envisoned not to be the right one, therefore humiliating themselves and their thought school.

The Fascists, as a team, failed. Their "ways of working", their "tools", did not survive. Conflict was shown not to be the "herald" virtue it had been advertised for.

The Fascist's (failed) endeavours is a perfect example as in how conflict is toxic for a team (and im sure everyone agrees on this); even if used as an external tool for "improving" the team's own wellbeing (you won't make good friends if you make everyone an enemy).

I've come to cherish different grades of "hostility" (if you pass me the term) whilst communicating through the years:

- disagreement

- questions

- (counter)argument

- agreement

- violent agreements

Each of these carry a different grade of severity; heaviness in being addressed; and difficulties for the approach.

A disagreement is the simplest form; i.e. any reason that is incapacitating people to find consensus / understanding / common ground.
Typical scenarios can variegate from hearing only "no", to offensive expressions flying in the air.

A question, as such, is a simpler and more neutral form. They might be part of disagreements, or then gather people towards consensus, which is where they get the neutrality from. On a question, you typically have an answer; but the difference is, the discussion ends there. The answer satisfies the need for knowledge completely, no other question (argument or counterargument) is necessary.

An argument, or counter argument, you can have presented either as a reason in a (civilised or uncivilised) discussion, or as a question, posed as an answer to another question, or a follow-up question to an answer.

An agreement is then the situation of (peaceful) "yes"'es which highlight the fact consensus has been achieved, and everyone (or at least the relevant parties) are on the "same page".

A violent agreement, finally, tackles down to an argument being agreed on by all the involved parties, in a "violent" fashion. Think of someone all being involved in a discussion on a given topic; but all shouting wildly "yes" at eachother, as if every one's "yes" would be better than the others (spoiler: they are all good the same, but the passion transpares here the most).

The maturity of a team (and the individuals composing it) can be seen from how "conflict"  situations are handled; more mature teams are more likely to not dig into the details of the issue, whereas younger, less mature teams will tend to split arguments up, hereby achieving the opposite effect, i.e. generating more tracktion for the respective cause - the conflict will spread. Mature teams will most likely steer clear from disagreements with the help of questions, to hover over into arguments, and from there to agreements (be them violent or not).



Saturday, October 22, 2022

Hardware-ology - how to restore a Vaio laptop to Win10

So I happened to get ahold of an old Vaio Laptop. No recovery disk. At time of use, came with Win7.

My kids have asked for a laptop for a long time for now, and I figured this is a good shot.

However.

The Vaio is pretty old (~2018 or such), and targeted to Win7.
The bios on it appears to NOT be able to boot from other than FAT32 partitions.
Win10 images come in UDF format (some industry craze of later times apparently),
which - you guess it - cannot be picked up by the bootloader on the Vaio (needless to say, the bios is as barenaked as possible).


After excluding also PXE (my home router does not support it); it seems there is time to take a different step. How about slicing out the extras from the Win10 installation image to make it fit on a 4GB partition? In the end, it's merely 1.8 GB over that limit. Shouldn't be that hard, right?

So I found this link from Dell that tells how to slice the windows image.

However.

I have been a debian user for about 7 years now, so there is no windows machine around at my home.   

So,  the best solution was to get one of the Edge Virtual machines Microsoft kindly provides, and hope the tool in question ("dism") is available...and guess what? On the Win10 VM, it is!

So, downloading the latest Win10 ISO, installing guest tools on the Edge VM, sharing a folder, correcting the file access rights to basically 777 to _everything_ and splitting the install with this command:


`dism /Split-Image /ImageFile:"c:\temp\install.wim /SWMFile:"c:\temp\install.swm" /FileSize:4000`

Great. That was easy. Now we have a Win10 ISO that fits on a FAT32 partition.

Unfortunately, the Vaio still does not comply with the USB stick and just display an empty screen with a flickering cursor.

The Vaio itself is not a USB-3 device. But the stick is 2.10.

Bummer.

So the next step is, if it's not a "legacy" boot device, it can only be a UEFI one, right?

Thanks to the Alpine people, over here we have perfect instructions on how to create a UEFI-compatible boot stick.

So, in this case, reformat the stick, and re-sync all the files (this has become tedious by now hasn'it it?).

Well, same result. Bummer #2

EDIT : long story short, downloading Rufus and installing the ISO on the usb stick was the straightforward solution. Done.
For some reason, the formatting of the stick is incredible specific for these old MBR/UEFI booting machines.

Sunday, August 29, 2021

The Barber paradox

The barber, a well-shaved man,  shaves all and only the people from a town who do not shave by themselves. Who shaves the barber?

A barber - as defined by the assumption - is someone who shaves all and only people who do not shave by themselves.
Furthermore, any individual is a barber, as long as he shaves more than one person. Let's assume for simplicity there to be only one barber.

"All" in the context of the statement is not defined as a number; but we can assume (for the sake of simplicity) for it to be more than one, i.e. at least two. The definition, furthermore, states also that there is more than one person (=people) that do not shave alone.

Lets assume all the villagers, including the barber, those who shave alone, and those who don't, to be individuals. Let's also focus only on the individuals who do not shave alone, and the barber, and forget the ones who shave alone.
Let's further include another assumption, i.e. that all individuals can shave.

Assuming we have only one individual acting as a barber all the time in the town (i.e. one single individual who shaves more than one, and all persons from the set that do not shave alone).


In the set of individuals, a barber stands out as an individual who shaves more than one person - in fact the barber shaves "all" the people who do not shave alone - except himself, which he cannot, due to definition (this btw precludes him being part of the set of people who do not shave by themselves). All other individuals can shave without being barbers, as long as everyone shaves only one and the same person - the barber.

All the people who are shaved by the single barber can therefore shave the barber; as the barber is a single individual; they do not have to be considered as barbers in the terms of the original definition.

This sets the barber into a "special" barber set - it shaves everyone; but is shaved by everyone else.
Moreover; the barber is not a barber, if he shaves himself.

Is this crazy or what?

Further developments:
- barber is a role that changes from individual to another (in turns; the question is, not who shaves the barber, but who is the barber today? ); i.e. can there be more than one barber in the town, for whom the same restrictions apply?


Friday, October 30, 2020

JIRA herp derp!

So you want to use jira, because it's fancy and great and awesome. Yes you are right, it is good (at least for sw development).
And you want to do it quickly; so you use a docker container?
And you want to use it safely; so you want HTTPS?
And you do NOT want to put it behind an apache reverse proxy?
Well, Tomcat supports https just well.
BUT! Keep in mind that the key store needs to be in JKS format,as the JVM is 1.8.xx. Yes, ladies & gents, in atlassian's docker container, the JVM is ancient.
I just spent three hours figuring out why "trustAnchor is empty". Herp derp