I've been bitten by this at least twice and i'd think others have too. I have a field expanded making changes and I click the "X" to collapse. Thankfully for the second screen confirm I catch myself. Why not just change it to the "trashcan" icon?
Great idea. That's definitely happened to me.