The same thing happened to me a year ago. I had an AWS account I barely used except for an SQS queue. Somehow, someone got in, changed the password, and set up a machine learning pipeline. I couldn't turn it off since I was locked out of the account and I racked up a $20k bill.
I also have no idea how they got it. I had 2FA set up and only had one service key created that I used in a Heroku environment variable.
Here's what happened:
- I contacted customer service. It took them several days to get back to me. Initially they told me they couldn't help and I would be responsible for any charges per their ToS as it's my responsibility to secure the account.
- After some back and forth, they reset my account credentials (the email was changed from me@mycompany.com to uuid@random.ru so it was obviously an account takeover).
- They listed out a list of services that had been started after the compromise and told me it was my responsibility to disable them and then tell them I did so.
- I cleaned things up the best I could and then told the service agent. They said I missed a few things and gave me more clear directions.
- By this point I had a $70k bill. Things had been running for about a week.
- I asked about getting a refund and they said they could do that but only after I set my account up with a proper security setup, which involved creating a bunch of separate small user accounts with minimal permissions.
- I did that, they refunded the charges, and then I deleted my account.
Long story short, it took a while and they weren't initially too helpful but ended up being very nice and helpful in the end.
Comments
The same thing happened to me a year ago. I had an AWS account I barely used except for an SQS queue. Somehow, someone got in, changed the password, and set up a machine learning pipeline. I couldn't turn it off since I was locked out of the account and I racked up a $20k bill.
I also have no idea how they got it. I had 2FA set up and only had one service key created that I used in a Heroku environment variable.
Here's what happened:
- I contacted customer service. It took them several days to get back to me. Initially they told me they couldn't help and I would be responsible for any charges per their ToS as it's my responsibility to secure the account.
- After some back and forth, they reset my account credentials (the email was changed from me@mycompany.com to uuid@random.ru so it was obviously an account takeover).
- They listed out a list of services that had been started after the compromise and told me it was my responsibility to disable them and then tell them I did so.
- I cleaned things up the best I could and then told the service agent. They said I missed a few things and gave me more clear directions.
- By this point I had a $70k bill. Things had been running for about a week.
- I asked about getting a refund and they said they could do that but only after I set my account up with a proper security setup, which involved creating a bunch of separate small user accounts with minimal permissions.
- I did that, they refunded the charges, and then I deleted my account.
Long story short, it took a while and they weren't initially too helpful but ended up being very nice and helpful in the end.
Jesus. This is terrifying.
Imagine if they had not reimbursed it!
No kidding. They told me they wouldn't reimburse me if it happened again which is why I deleted my AWS account and moved to something else.