Last night, I was walking home from work wearing my new raincoat. This thing is incredible. It's a Gortex shell that is very waterproof. So much so that I didn't even realize it when it started raining. You'd think that's a good thing, right? But my briefcase, which was slung over my shoulder, got soaked! I had an umbrella, but not realizing that it was raining I never opened it up.
This isn't just a story on my foolishness. There are many examples of fantastic design that insulate the user from what is going on outside and cause him/her to lose important situation awareness. I read once of a plane incident where the autopilot adjusted so seamlessly to a lost engine that the pilot didn't know that it was lost. When he took over the controls to land the plane, it veered sharply to the left (2 engines on one side, only 1 engine on the other side). Without some quick thinking, he might have crashed. I am sure he had some nauseated and frightened passengers.
So what is the lesson of the story? There is a user requirement that we often forget about: transparency in automation. When designing automation, make sure that it isn't so good that the user loses important features of situation awareness. I have heard of workers who monitor an automated production process and after a few years of Six Sigma performance they completely forget how to run it manually. If/when the automation does fail - the line is shut down.
Don't let this happen to you!