I want to share an anecdote of confusing programming language features as a caution to my fellow programmers.
Reviewing our application logs, I noticed that the API requests were interspersed with pauses of irregular durations. The interval between each successful request is supposed to be fixed; and the interval between each failed request, exponentially growing up to a configured maximum. For example, after the first failure, the application pauses for 5 seconds; after the second, 10 seconds; after the third, 20 seconds; and so on. However, the logs showed intervals of 15 seconds, 30 seconds, 60 seconds, then 0 second—this was odd, especially the last 0 second.
This behaviour remained a mystery even after a colleague helped me look for errors in our C# code. We reviewed this line of code together but found nothing wrong; it clearly showed that on every retry the base interval was doubled.
int tmp = RetryIntervalInMs * (2 ^ retryCounter);
By coincidence, I needed the same calculation of intervals in my hobby embedded project. As I started to write the code that evening, I made an epiphanic realisation.
In our C# code, we had used the
^ operator, interpreting it as raised to the power of. This is true in some languages – BASIC, for example – but not in C#, where it is actually the operator for XOR. I was amazed that this mistake escaped our scrutiny..
Once we had identified the cause, it was easy to fix the error by using the correct library method
int tmp = RetryIntervalInMs * (int)Math.Pow(2, retryCounter);