Property based testing is your robot army

June 29, 2017 / / 0 Comments

Let’s get the formality out of the way – what is property based testing

In the very narrow definition, a property based test is comparing and verifying that a program’s output complies with a rule (property), rather than a specific value.
That being said, add this to the mix (Taken from scalatest.org):
“…a property is a high-level specification of behavior that should hold for a range of data points”.

Translation

Property based testing involves testing your code’s output against one (or more) constraints, for one (or more) input values.

In real life

Property based testing is usually used by running multiple, randomized (or otherwise entered by the test developer) input values, and matching the output values to a set of business rules.

Why do I need this in my life?

  •  All odd numbers are prime – Random data is (sometimes) smarter than you. Each test run gives you multiple values, that to some degree, were never used before. Some of them are exactly the secret key needed to break the code.
  • Real world data is your enemy in production, but your friend in testing – Dumping a load of production derived input values tests your code against the threats that it is expected to meet in real life (see here).
  • Unknown unknowns – What you think is wrong with the code (known) should be fixed on the spot. Areas that may hold issues (unknowns) should be tested specifically. Dangerous areas that you don’t know that you don’t know about (unknown unknowns) will not be encountered unless you cast a very wide net, spanning more than you can see. Combining a multitude of input values with many constraints will help you stumble upon them.
  • “The only doctor that could have saved me was me” – Sometimes, producing the specific expected value is too resource costly, or only possible using the very same code you are trying to test (nullifying the effort).

Continue reading

Black Swan (by Nassim Taleb) extreme book review / summary

November 23, 2016 / / 0 Comments

SELECT DISTINCT (Idea) FROM NassimTaleb.dbo.BlackSwan

 

  1. People are crap at making predictions.
  2. People are crap at knowing they are crap at making predictions.
  3. One of the main reasons that people are crap at making predictions is that they are using thought and mathematical models that rely on moderate / capped input variation, as can be encountered in the physical world (e.g. the variation in the height property in a group of individuals). However, when the input varies wildly (e.g. distribution of wealth) there are extreme edge cases.
  4. Another reason for the crappy prediction ability is the existence of “black swans”, events that are not foreseen and cause massive effect.
  5. Some try to confront the black swan threat by brainstorming and allowing for extreme cases in their models. This will not work, as the black swans are by definition the events not foreseen.
  6. While the black swans cannot be foreseen, their very existence can be, and the wise will hedge herself against the effect, even while not predicting the specific cause.
  7. Academics and theoreticians believe that they are superior to practitioners. They are wrong, since practitioners are way better.
  8. The author is a practitioner.
  9. Ignoring the above, the financial world and the Nobel prize committee tend to prefer theoreticians, staking it all on models that crumble in the face of real world events (at great cost).
  10. While not providing a fluent reading experience, there are several ideas I will be taking with me. All in all, I’m happy that I invested the time reading this book.

Alice’s adventures in startup land

July 17, 2016 / / 0 Comments

Disclaimer

Had I been Mr. Liddell, even when considering the different artistic values of the 1850s, Charles Dodgson would not have lived to hear of the end of the crimean war. Maybe, if I contained myself, he could “just” spend the rest of his life sitting down for his No.1s.
All this aside, “Alice’s Adventures in Wonderland” is a very interesting read, which takes some stabs at the grown up world, while keeps a healthy dose of cuckoo bananas.

Never made the New York times best sellers list

It’s been more than 150 years (OK, 151, which is indeed more than 150) since Alice’s Adventures in Wonderland was first published. During that time, the text and the original drawing achieved meme state, but also the derivatives (Disney movie, Jefferson Airplane song) originated multiple memes. Let’s see if we can map our high tech life to Alice quotes, as we already do to other canons as the bible, 9gag or Seinfeld.

Continue reading

Boolean logic mistakes ruining projects since the 17th century

April 14, 2016 / / 0 Comments

No test for the wicked

In 1631, the respectable duo of London royal printers, Robert Barker and Martin Lucas were commissioned to print a new version of the King James Bible. Almost to perfection, they did a wonderful work. Actually, considering the hundreds of thousands of words the book contains, omitting just one is pretty good. Unless, of course, it is placed smack right in the middle of the seventh commandment, and the word omitted is “not”. In this case, you get a 180 degree turn on the better accepted meaning of the commandment, which is now “Thou shalt commit adultery”.

Continue reading

Without proper QA metrics we are just pathetic useless ants

February 10, 2016 / / 0 Comments

Things that goes crunch in the night

The contract between me and the ants that frequent my apartment is pretty straightforward:

  1. As ants, you get to keep whatever you touch.
  2. Seriously, if you touch it, please take it away.
  3. I, as human, get to dispose of any ant I see, in any form I choose, depending on my mood and free time.
  4. If you play nice, the bug spray will stay at the locked cabinet, and we can all go about our respective businesses.

Now, let’s examine a case where a crumb of sorts (OK, 99.7% that it’s a single corn flake) is left overnight on the floor.

Expected result:

  1. Come morning, no crumb on the floor.
  2. No ants on the floor.

 

Actual:

  1. A convoy of 300 ants stretches across the apartment.
  2. On one side – that damn corn flake.
  3. On the other side – a disarray of confused ants, going in all directions.
  4. The corn flake is exactly where it fell on the floor the day before.
  5. Corn flake is surrounded by 50 ants or so. Half of them are already dead (out of shame, if there is some decency in them), the rest seems to be pushing and pulling, but getting no actual movement out of the corn flake.

Continue reading

The boss’s nightmares are your ticket to get hired for a QA job

January 10, 2016 / / 0 Comments

 

To keep the balance you have let some dark in

It might sound a bit Sith, but playing your best traits is not the only thing that will land you that QA job. Usually, especially when the interviewer is not a QA person (which probably makes him / her a development leader of sorts), triggering one of the common fears is an immediate way to get disqualified. If you can address these fears (even when not specifically asked about it), you gain some very helpful points.

Continue reading

This changes nothing

October 13, 2015 / / 0 Comments

This changes nothing, probably

It is unavoidable, but sometimes, right before a product deployment (or just after), a patchable issue is found. Not something that merits a full version, something simple, isolated. Could be a completetly new feature, which should not impact the current user experience in any way. Right?

This is also something that QA professionals should learn to live with. You score points by not backing down, as long as it helps the company, not by insisting on a week long regression after changing a typo in the GUI.

Continue reading

Four rookie mistakes that will botch your first job interview

August 16, 2015 / / 0 Comments
Your own worst enemy

Landing your first job or two is usually the hardest. You have zero experience, no referrals and no connections. Wouldn’t it be a pity if on top of all that, you will also get in your own way? When recruiting for a junior position, the interviewer has so little to work with, that the slightest fault is enough to differentiate you (for the worse). On the bright side, if you can avoid steamrolling over yourself, you have a starting point higher than some of your competitors. You can start by avoiding these self sabotaging ideas:

1. Listing experience and knowledge in your résumé, that is just not true

As I already mentioned , reliability is the QA person’s most important characteristic. If your CV claim something you cannot backup in the interview, any tests you might be surprised with, or even during your employment, there goes your reliability.
This includes listing things that you once knew, or had very limited experience with. Sometime during the last century (while dodging saber tooth tigers), I completed my first university course in Pascal. Sure, it will be technically true to claim experience in Pascal, but the truth is, that the time it will take me to regain competence in Pascal is more or less the same as it will take me to pick up a simple language I don’t know at all. It will be just as dishonest to claim experience in something I forgot ages ago, as it is to claim experience in a programming language I never learned. Almost all the CVs I get for junior QA engineers claim some experience in SQL. Most of them cannot differentiate between inner and outer JOIN.

Continue reading

Qualitygate – the scandal you should care about

July 25, 2015 / / 0 Comments
Tiger Tiger

Imagine a dangerous tiger escaping the zoo and roaming through town. After a chase full of heroics and close calls (miraculously no one was hurt), the beast is caught and placed back in its old cage. Now guess what the town square conversation is about. “How did the tiger escape the cage?” seems a solid bet. “Even if it somehow manage to escape the cage, how did it exit the zoo?” Is another valid question. Since the folk of this specific town are both responsible and well spoken, one can expect: “which additions are required to our tiger prevention means in order to prevent future escapes?”.
One might think that in the high stakes, highly optimized world of software development, the same logic will apply. But for some reason, when a bug is found, most of us feel pretty good if we can fix it, get the tiger back to the same cage, behind the same moat and the rest of the defenses that failed before.

And sooner or later, the tiger gets peckish again…

Continue reading

« Older posts

Categories

© 2024

Theme by Anders Noren — Up ↑