Today has been very interesting to say the least. My team was testing an environment that was just cloned from production. We are making sure nothing is super broken. Well, no one could get past one screen without a Help Desk Error. So, it was looking like an environment wide issue. Little did we know, our developers were noticing a problem IN production that was causing interfaces to fail. A few email chains later, it came out that a different team pushed a change to production without testing and it failed across the board. The dev team supervisor isn't in office but I can feel him seething through his emails. My supervisor laughed because this is such a ridiculous problem that shouldn't be happening. There is a QA team for a reason. We should be able to catch these problems earlier and prevent this s#*! show. Oh boy.
UPDATE:
I'm writing this in an email to myself as it is all happening. Basically, this change has affected all environments and we are stuck. A patch could be made to bandage the problem briefly but I'm not sure if our team works like that. I'm still learning as I go along. Currently, its mid-morning and supervisors are scrambling. No one can do anything except maybe work on test cases. I just started a new audiobook so I'm quietly listening to that and waiting this mess out.
UPDATE 2:
Its mid-afternoon. While working on some version testing scripts, I received an email from our interface developer to try the software again. I was able to get it to work through a transaction. I get to finish my work today! I hate leaving incomplete work that I should have been able to get done in the first place for the next day. Off I go!
No comments:
Post a Comment