Quora Internships: What is the most catastrophic mistake made by an intern at a tech company?
OMG these are great stories
http://www.quora.com/Internships/What-is-the-most-catastrophic-mistake-made-by-an-intern-at-a-tech-company
Jeff Nelson, Invented Chromebook. Former Go... (more)
Ken Brody, Ex physicist, computer scienti... (more)
Jay Bazzinotti, No one is perfect... but I am ... (more)
Ramesh Kaika, Solver of Problems and sometim... (more)
Gil Silberman, Lawyer, technologist, social s... (more)
I was an intern at SlideShare . My project was to come with an 'admin panel'. The admin panel was essentially to serve as a one stop tool for creating, suspending, deleting, reconverting slideshows (and in some cases deleting the users). Most of the tech companies use a Master-Slave database. So all the reading operations are performed via the 'Slave' and the Master is supposed to handle all the 'write' operations. Master-Slave are in sync.
'Reading' operations can involve running a query such as to get the user's password, last login etc. 'Writing' operations would involve 'writing' the new password to the database, 'writing' the slideshow title uploaded and so on.
My code was supposed to 'read' all the user info. I had made a paginated interface limiting the user count on each page to be 50. The next 50 users should be fetched only if you navigate to the next page, else you can type the username in a search box to fetch that particular users' info. So in principle the query should be run on a 'slave'. Instead I ran this query, and yes I ran this query on the Master ( which is too busy doing the writing operations already!) :
SELECT * FROM users ;
I don't remember the exact user count then, but to add to this, I refreshed the page multiple times which used to perform this query.
SlideShare was a 503[Service Unavailable] for around 10 minutes.
Jason Perlow, Sr. Technology Editor, ZDNet. ... (more)
Suggestions Pending
Amin Ariana, Technical Founder, hacker and ... (more)
Jeff Kesselman, 25 years in the video game ind... (more)
OMG these are great stories
http://www.quora.com/Internships/What-is-the-most-catastrophic-mistake-made-by-an-intern-at-a-tech-company
Jeff Nelson, Invented Chromebook. Former Go... (more)
575 upvotes by Hong Yi Dong, Alexander Blekh, Johnny Zavala, (more)
Not at a tech company, either, (maybe need a new question for student screwups?) when I was back in college as an undergrad, I was hired as a research assistant mostly to build numerical models for a physics lab. As a kind of reward, they flew me out to Fermi Lab for a week to "help with" the actual experiments - as if an undergrad could really contribute in any meaningful way to a billion dollar research facility.
So, my professor came up with a project to wire up some NIM logic cabinets for one of the experiments, which is really quite boring. It's just plugging in calibrated cables of different types and sizes according to a diagram; a bit like setting up a data center, if a data center had 20 different types of cables that all had to go in particular plugs on the front panel.
What I didn't know, the experiment was live! The cabinets were live! Worst of all, the fire suppression system was live!
I went about my business for about two hours, when I bumped the fire suppression system.
(BBBUUZZZZZZZZzzzzzzooooo
Having set off the fire alarm, the automatic emergency protocol was to kill all power to everything, including the experiment (a 4 mile circumference particle accelerator).
A billion dollars in equipment and about 100 Physics PhDs come to a screeching halt.
My professor comes running into the room with an alarmed look on his face. He points out the little tube running along the bottom of the cabinet that triggers the fire suppression system and explains what happened.
"You bumped that. The whole experiment is shut down. It's going to take 45 minutes to bring everything back online."
"Oh... so... should I finish wiring the cabinet?"
So, my professor came up with a project to wire up some NIM logic cabinets for one of the experiments, which is really quite boring. It's just plugging in calibrated cables of different types and sizes according to a diagram; a bit like setting up a data center, if a data center had 20 different types of cables that all had to go in particular plugs on the front panel.
What I didn't know, the experiment was live! The cabinets were live! Worst of all, the fire suppression system was live!
I went about my business for about two hours, when I bumped the fire suppression system.
(BBBUUZZZZZZZZzzzzzzooooo
Having set off the fire alarm, the automatic emergency protocol was to kill all power to everything, including the experiment (a 4 mile circumference particle accelerator).
A billion dollars in equipment and about 100 Physics PhDs come to a screeching halt.
My professor comes running into the room with an alarmed look on his face. He points out the little tube running along the bottom of the cabinet that triggers the fire suppression system and explains what happened.
"You bumped that. The whole experiment is shut down. It's going to take 45 minutes to bring everything back online."
"Oh... so... should I finish wiring the cabinet?"
Not a tech company, but about 18 years ago as a student, I was doing research at London's Heathrow Airport, working in a room just below the control tower.
We were listening to the control tower instructions and timing how long the pilots took to respond.
My colleague left for lunch one day having turned his radio onto 'broadcast' by accident. As radio is one way, it meant that no one was able to send or receive messages on the frequency that was being used to give take off permission.
I returned back to my desk and started to eat my lunch to discover that all departures from the airport had been brought to a standstill by someone who sounded like they were eating their lunch.
The realisation that it was the sound of MY lunch being eaten hit me about 10 minutes later.. I rushed over and flicked the switch to off, and one of the busiest international airports in the world started to work again.
We got away with it though.
We were listening to the control tower instructions and timing how long the pilots took to respond.
My colleague left for lunch one day having turned his radio onto 'broadcast' by accident. As radio is one way, it meant that no one was able to send or receive messages on the frequency that was being used to give take off permission.
I returned back to my desk and started to eat my lunch to discover that all departures from the airport had been brought to a standstill by someone who sounded like they were eating their lunch.
The realisation that it was the sound of MY lunch being eaten hit me about 10 minutes later.. I rushed over and flicked the switch to off, and one of the busiest international airports in the world started to work again.
We got away with it though.
Ken Brody, Ex physicist, computer scienti... (more)
176 upvotes by Leonid S. Knyshov, Ilya Nepomnyashchiy, Alexander Blekh, (more)
I was a newbie physicist working in a physical chemistry lab in Boston. My desk sat next to Dr. Krainian (not his real name) so we were face to face eating lunch that day, with our desks between us. I noticed a heavily sealed jar on his desk.
"What's that?"
"Well, you know I've been having problems with the local dogs peeing on my bushes, so I got something to fix it."
"Is that the jar?'
"Yep. Methyl mercaptan, ISO."
That is the primary ingredient of skunk smell, but skunks don't use the pure stuff. We used it as an odorant in certain gases.
While I was trying to eat my sandwich, the curious Dr. was sniffing the jar. Apparently he couldn't smell anything. Inside the sealed glass jar was a block of paraffin. Inside the paraffin was a small stoppered vial. Inside the vial was the skunk juice.
By the time I was on my dessert, Dr. Kranian had opened the jar. I saw him chipping away at the block of paraffin. He chipped a bit, sniff, nod and frown, then chip a bit more. I just hurried to finish my meal. The lab had that kind of a reputation, usually with large tanks of highly compressed gas, fragile glassware in precarious places, and other lab mayhem.
As I was headed for the exit I saw him chip away the last of the paraffin and verrry carefully, verrry slowly, crack open the vial. A whiff escaped. His eyes bugged out, he coughed and HE DROPPED THE VIAL!
That whole wing of the building was evacuated and cleaned and cleaned and cleaned and it was quite a while before anything other than sniff tests were conducted there.
"What's that?"
"Well, you know I've been having problems with the local dogs peeing on my bushes, so I got something to fix it."
"Is that the jar?'
"Yep. Methyl mercaptan, ISO."
That is the primary ingredient of skunk smell, but skunks don't use the pure stuff. We used it as an odorant in certain gases.
While I was trying to eat my sandwich, the curious Dr. was sniffing the jar. Apparently he couldn't smell anything. Inside the sealed glass jar was a block of paraffin. Inside the paraffin was a small stoppered vial. Inside the vial was the skunk juice.
By the time I was on my dessert, Dr. Kranian had opened the jar. I saw him chipping away at the block of paraffin. He chipped a bit, sniff, nod and frown, then chip a bit more. I just hurried to finish my meal. The lab had that kind of a reputation, usually with large tanks of highly compressed gas, fragile glassware in precarious places, and other lab mayhem.
As I was headed for the exit I saw him chip away the last of the paraffin and verrry carefully, verrry slowly, crack open the vial. A whiff escaped. His eyes bugged out, he coughed and HE DROPPED THE VIAL!
That whole wing of the building was evacuated and cleaned and cleaned and cleaned and it was quite a while before anything other than sniff tests were conducted there.
I sat at the MySQL console for the production web application database and typed:
...and hit return.
Note that I typed
, not
.
In MySQL the statement (
) is
if it evaluates to
, and
if it evaluates to anything else. I just asked the database to delete all rows where the ID minus 1234 was not zero.
It replied "80,000 rows deleted".
DELETE FROM tablename WHERE id - 1234;
...and hit return.
Note that I typed
id - 1234
id = 1234
In MySQL the statement (
value - value
false
0
true
It replied "80,000 rows deleted".
Jay Bazzinotti, No one is perfect... but I am ... (more)
633 upvotes by Diana Tan, Abhiram Chivukula, Leonid S. Knyshov, (more)
I worked at a company that helped propel the internet in the early days by inventing the error-correcting modem. This product revolutionized computer transmission of data because for the first time, medical, financial, business and other data could be transmitted reliably. The implications were enormous. Today we take this sort of thing for granted, like breathing, but in those days it was a world-shifting phenomenon.
In order to stay ahead of the competition, who were licensing our technology, we were always enhancing our techniques. If you are now sending perfectly reliable data, then the next thing is to greatly increase the transmission speed. We implemented a patented version of Huffman data compression that was breathtaking in its simplicity and power. The device would "guess" what the next character was going to be before it was received based on a frequency analysis of previous data, and then use a shorter bit pattern for that data. We were able to double transmission speeds. Then we did it again using a Markhov compression technique. We were awesome and impressive and the industry was amazed.
We often hired interns, and we treated them as equals, something that didn't happen often in industry. We didn't send them out for coffee -- we immersed them in the work and rewarded them when they contributed. We had one intern we particularly liked. He was quick and bright, and he provided real contributions to the technology. Then one day, the FBI swarmed down on our company. They came in fast and hard and all work came to a stop while we wondered what the hell was going on. It turned out that our intern had stolen the source code for our most precious and valuable techniques and tried to sell them to the Chinese for the pitiable sum of $50,000.
Unfortunately, the Chinese he was selling them to were undercover FBI agents. I never discovered how he set this sale up or how the FBI became involved, but here was a young man, not even a graduate of college, who was now heading off to prison for espionage, theft, and a dozen other charges, a man who would never graduate from college and who would never get a job in the industry he was fully trained to support and who was talented and capable. His life was ruined and as I said, the amount of money he was seeking was ridiculously small considering the technology made us 100s of millions of dollars. I would say this rates right up there with the biggest intern gaffes in history.
In order to stay ahead of the competition, who were licensing our technology, we were always enhancing our techniques. If you are now sending perfectly reliable data, then the next thing is to greatly increase the transmission speed. We implemented a patented version of Huffman data compression that was breathtaking in its simplicity and power. The device would "guess" what the next character was going to be before it was received based on a frequency analysis of previous data, and then use a shorter bit pattern for that data. We were able to double transmission speeds. Then we did it again using a Markhov compression technique. We were awesome and impressive and the industry was amazed.
We often hired interns, and we treated them as equals, something that didn't happen often in industry. We didn't send them out for coffee -- we immersed them in the work and rewarded them when they contributed. We had one intern we particularly liked. He was quick and bright, and he provided real contributions to the technology. Then one day, the FBI swarmed down on our company. They came in fast and hard and all work came to a stop while we wondered what the hell was going on. It turned out that our intern had stolen the source code for our most precious and valuable techniques and tried to sell them to the Chinese for the pitiable sum of $50,000.
Unfortunately, the Chinese he was selling them to were undercover FBI agents. I never discovered how he set this sale up or how the FBI became involved, but here was a young man, not even a graduate of college, who was now heading off to prison for espionage, theft, and a dozen other charges, a man who would never graduate from college and who would never get a job in the industry he was fully trained to support and who was talented and capable. His life was ruined and as I said, the amount of money he was seeking was ridiculously small considering the technology made us 100s of millions of dollars. I would say this rates right up there with the biggest intern gaffes in history.
Ok I give in because I got an A2A -- I don't really consider this catastrophic, but it's a funny story and I had people messaging me from all around the company telling me what a great laugh they had encountering this bug (including the team that had to fix it!):
I joined at the beginning of my internship under the username www(they are my initials), and all the engineers' sandboxes started to redirect to mine because of the url address.
So yes, I managed to break things without even writing a single line of code.
I joined at the beginning of my internship under the username www(they are my initials), and all the engineers' sandboxes started to redirect to mine because of the url address.
So yes, I managed to break things without even writing a single line of code.
Ramesh Kaika, Solver of Problems and sometim... (more)
300 upvotes by Nimit Sohoni, Lawrence Gimenez, Leonid S. Knyshov, (more)
I was 21 and at a multinational IT consulting firm. Target was one of our clients, and at that time, used the IBM dumb terminals for communication. The 'Message Send' command was used similar to IM now-a-days, but if you do not provide an id, it sent the message to ALL users (which few people knew about)
One Friday I sent, what I thought was a joke - 'To commemorate Blaise Pascal's birthday, all your computers will be run at half-speed today'. I thought it sent the message to all IT folks, but what I did not realize was that the message was sent to ALL uses - retail stores, Target business users, management teams etc. And that Friday turned out to be the 13th. So Target security thought it was a virus attack, and traced the origin of the message to, natually, my cubicle. I was called into work by my CTO at 2 AM, and I, with a pale face, explained how it was supposed to be a harmless prank. Then the CTO and with the CEO on the phone crafted an apology letter to the Target CEO/CIO with me right there. I was pretty sure I was fired, but I wasn't.
On the plus side I became famous and made lot of new friends in the company :). And Target implemented a change to the IBM send message to prevent this.
One Friday I sent, what I thought was a joke - 'To commemorate Blaise Pascal's birthday, all your computers will be run at half-speed today'. I thought it sent the message to all IT folks, but what I did not realize was that the message was sent to ALL uses - retail stores, Target business users, management teams etc. And that Friday turned out to be the 13th. So Target security thought it was a virus attack, and traced the origin of the message to, natually, my cubicle. I was called into work by my CTO at 2 AM, and I, with a pale face, explained how it was supposed to be a harmless prank. Then the CTO and with the CEO on the phone crafted an apology letter to the Target CEO/CIO with me right there. I was pretty sure I was fired, but I wasn't.
On the plus side I became famous and made lot of new friends in the company :). And Target implemented a change to the IBM send message to prevent this.
It was my second year in university and I worked for a local ISP as a junior system administrator. We had a large wireless network ~60km in radius covering our whole city and many rural areas around it. This network was used by all major commercial banks and many large enterprises in the area (~100 of bank branches, some large factories, radio stations, etc).
To cover such a large area (in Ukraine in 2002), about 50% of which were rural villages and towns, we basically had to build a huge wifi network, that had a very powerful antenna in the center and many smaller regional points of presence would connect to it using directional wifi antennas and then distribute the traffic locally. The core router connected to the central antenna was located at the top floor of the highest building in the area about 20 min away from our central office.
One day I was working on some monitoring scripts for the central router (which was a pretty powerful unix server). I'd run those scripts on a local server that I had on my table, make some changes, run it again, etc. While doing my debugging I received an alert from our production monitoring saying that our core router had some (non-critical) issues and, since I was on-call that day, I've decided to go take a look. After fixing the issue on the router I went back to my debugging and successfully finished it within an hour or so.
And that's where things went wrong... When I wanted to shut down my local server I've switched to a terminal window that was connected to the box, typed "poweroff", pressed enter and only then realized that I did it on a wrong server. I had that second window opened ever since the monitoring alert an hour ago and now I've shut down the core router of our city-wide network.
We had to grab a car and drive to the central station to power the router back on, our whole banking infrastructure was down for 30+ min and that was one of the darkest days of my career.
To cover such a large area (in Ukraine in 2002), about 50% of which were rural villages and towns, we basically had to build a huge wifi network, that had a very powerful antenna in the center and many smaller regional points of presence would connect to it using directional wifi antennas and then distribute the traffic locally. The core router connected to the central antenna was located at the top floor of the highest building in the area about 20 min away from our central office.
One day I was working on some monitoring scripts for the central router (which was a pretty powerful unix server). I'd run those scripts on a local server that I had on my table, make some changes, run it again, etc. While doing my debugging I received an alert from our production monitoring saying that our core router had some (non-critical) issues and, since I was on-call that day, I've decided to go take a look. After fixing the issue on the router I went back to my debugging and successfully finished it within an hour or so.
And that's where things went wrong... When I wanted to shut down my local server I've switched to a terminal window that was connected to the box, typed "poweroff", pressed enter and only then realized that I did it on a wrong server. I had that second window opened ever since the monitoring alert an hour ago and now I've shut down the core router of our city-wide network.
We had to grab a car and drive to the central station to power the router back on, our whole banking infrastructure was down for 30+ min and that was one of the darkest days of my career.
I brought down 3 floors of AIG’s Wall Street corporate network – for 3 days! This is a true story.
I was an entry-level network administrator at AIG on Wall Street in the early 1990’s. They just got a new CIO with new IT management. They were quickly changing everything. The team of network engineers was laid off. They wanted to move from Ethernet to Token Ring; from PCI servers to MicroChannel. All of this was happening very fast.
New management wanted to bring in ATT/NCR servers running the then new Windows NT. ATT configured a server for us to test. They sent it to me on a Friday.
Before I left for the long weekend, I decided to plug it in put it under my desk. The startup wizard asked me for the computer name. Since we didn’t yet have any naming conventions for Windows NT, I just called it “CU-NT01” – CU are my initials, this was our first NT box – made perfect sense.
Little did I know that the server was a DHCP server. Put 2 DHCP servers on the same network and they’ll create a situation where all the clients get confused when they renew their leases. In a nutshell, my new server created so much traffic that nobody could use the network.
Unfortunately, the network ops guys had no way to predict, isolate or even identify which of the nodes on the network was creating the problem. They knew the server name, but didn't know where it was! After 2 days, they had to use a network “sniffer” to find the offending machine. It took them most of the weekend and part of Monday to find it.
When I came in Tuesday morning, the CIO and several other big-wigs were hovering over my cube. It was not a pleasant week! But the whole debacle was the butt of many jokes for a long time.
I was an entry-level network administrator at AIG on Wall Street in the early 1990’s. They just got a new CIO with new IT management. They were quickly changing everything. The team of network engineers was laid off. They wanted to move from Ethernet to Token Ring; from PCI servers to MicroChannel. All of this was happening very fast.
New management wanted to bring in ATT/NCR servers running the then new Windows NT. ATT configured a server for us to test. They sent it to me on a Friday.
Before I left for the long weekend, I decided to plug it in put it under my desk. The startup wizard asked me for the computer name. Since we didn’t yet have any naming conventions for Windows NT, I just called it “CU-NT01” – CU are my initials, this was our first NT box – made perfect sense.
Little did I know that the server was a DHCP server. Put 2 DHCP servers on the same network and they’ll create a situation where all the clients get confused when they renew their leases. In a nutshell, my new server created so much traffic that nobody could use the network.
Unfortunately, the network ops guys had no way to predict, isolate or even identify which of the nodes on the network was creating the problem. They knew the server name, but didn't know where it was! After 2 days, they had to use a network “sniffer” to find the offending machine. It took them most of the weekend and part of Monday to find it.
When I came in Tuesday morning, the CIO and several other big-wigs were hovering over my cube. It was not a pleasant week! But the whole debacle was the butt of many jokes for a long time.
This is not related to a tech internship, but is still a bone-head move by an intern that still gets brought up years later.
The CPA firm I worked for always hired 4 or 5 summer interns and they just got picked up to help on miscellaneous engagements as needed. When they didn't have anything to work on, they were required to email the entire department with an "available for work" email so people would know they were free.
When one of the interns sent their email, another intern friend of his accidentally hit "reply all" with three simple words: Suck my dick.
The intern just told an entire department of 60-something people including partners, managers, department heads, etc. to "Suck it."
It was magical. Everyone got the email at the same time and you could see heads start to come up over cube walls one by one like prairie dogs. Managers slowly stepped out of their offices and everyone just stared at each other in shock.
They now offer "reply all" training as part of intern orientation.
The CPA firm I worked for always hired 4 or 5 summer interns and they just got picked up to help on miscellaneous engagements as needed. When they didn't have anything to work on, they were required to email the entire department with an "available for work" email so people would know they were free.
When one of the interns sent their email, another intern friend of his accidentally hit "reply all" with three simple words: Suck my dick.
The intern just told an entire department of 60-something people including partners, managers, department heads, etc. to "Suck it."
It was magical. Everyone got the email at the same time and you could see heads start to come up over cube walls one by one like prairie dogs. Managers slowly stepped out of their offices and everyone just stared at each other in shock.
They now offer "reply all" training as part of intern orientation.
Edit: Since this was published in Forbes there's been a lot of questions about whether this is a "screw up" and a few other things. I've added comments addressing this at the end of the story.
---
When I was 21 I almost lost several hundred million dollars by threatening to mutilate one of our customers.
In my senior year in college I worked full time as an intern PM at NetApp. I spent most of that time at work being groomed and prepared to be a full product manager, and given that my background was in cryptography I got pulled into a lot of customer meetings related to security.
One of our customers at the time was undergoing a big change with their security architecture, and I tagged along with one of the directors to the meeting. I was one of ten PMs giving talks on roadmap and our plans, and I had 30 minutes to convince their CIO and CEO that we could integrate our new systems well with the new security infrastructure they were rolling out.
It turned out though that the CIO and CEO weren't the only ones in the room. Joining them was the company's Chief Security Officer (CSO). Like me he was a young, rising star in their company with a lot to prove in a short period of time. He also didn't like me much from the get-go; when I walked in the room he sneered, and when I went to plug in my laptop to the projector he openly asked, "Is he really going to present alone?"
Most of my 5 slide presentation was instantly ripped apart. I had a good command of the tech involved, so the criticism wasn't on our findings. Instead, he nitpicked the design - the colors were off, the fonts weren't like the other presenters' (admittedly I did disregard the style designs, my bad), etc.
When I finished my slides and hit the time for questions, he laughed and shooed me away. "Good effort, but you clearly don't abide by our security practices." I stared at him with tired, dagger-piercing eyes across the podium. Not only did we abide by what they needed, but I'd spent all night working on this presentation (which combined with going to school full time meant that I was on very little sleep). I was pissed off, and I decided to push back.
Me: "Well what specifications are you referring to?"
Customer: "You don't understand. We are subject to a vast amount of compliance requirements inclu-"
Me: "-ding FIPS 140-2, PCI-DSS, FISMA..."
(I did my homework on the account)
Their CEO took notice at me pushing back and seemed to wake up from his "I don't care, when's lunch" stupor. As the CSO and I nerd battled like we were Sith and Jedi LARPers at Gen Con, a bunch of the account reps in the back of the room tried to get me to come off stage. My director let me stay.
Finally once I had proven that we fit the spec, the CSO changed tone to something ridiculous.
Customer: "Well what about biometric scanners? We need biometric scanners."
I blinked. Biometric fucking scanners? We're a storage company, not the Goddamn NSA. I responded that our authentication schemes supported most of the protocols that are used by bio-scanners, but he retorted that it needed to be first-party only.
I sighed, clearly exasperated, and responded bluntly. "We don't make biometric scanners. You don't need biometric scanners. They're expensive and none of your compliance requirements need them. It's complete overkill." The CSO immediately (and vehemently) shot back angrily, citing his military experience and how he was going to make an infrastructure that was "unhackable."
So I decided to turn the tables on him. "Okay, biometric scanners - what kind of biometric scanners do you need?" He gave me a basic list of specs, but having recently completed a homework assignment in my information security class (a class taught by a ex-NSA cryptanalyst who liked to talk about now-public faults in old security systems) on the topic I hit him back with the various faults in modern bio scanners - including the gory details on how you fraud them.
Me: "So you want bio scanners with feedback right? That's cool. Well what's to stop me from cutting off your thumb and swiping it like in the movies? Nothing. Unless of course you want to integrate temperature and humidity monitors, and even then I'll defeat it by running tubes into your cut-off thumb with warm water or soak it salt water in the microwave-"
At this point the CSO sat back horrified. The CEO of the company was dumbstruck, the account team in the back of the room was mortified, and my director was dying of laughter.
I proceeded to then go into detail about the faults of various retina scanners ("well I could pull out your eye and put a layer of Visine over the retina..") until finally the CSO sat back in his chair - both defeated in his designs and horrified at the glasses-wearing Asian kid in front of him that looked less like a brainy engineer and more like the Unabomber.
At this point the team decided to call it. I was quickly hurried out of the room - only after thanking the customers for their time of course - to where I met with the rest of the PMs in attendance who were literally doubled over with laughter. The account rep on the team later joined us and blasted me with a series of insults, noting that my insubordination might cost them the account and that I "clearly wasn't mature enough for my job."
I spent the next few days calling myself an idiot and getting ready to change my LinkedIn status. But when the email feedback report on my presentation came back I got the highest rating from the account's exec team. They noted that the CSO can "be difficult sometimes" and they appreciated that I "had a strong command and understanding of the security requirements of our space."
I spent the next few years at NetApp running product security. This event definitely came up during my year end review though, and since then I've become much better at presentation etiquette.
------
This is a screw up (note: original title was "fuck up") for a few reasons:
1.) You should never, NEVER, talk to customers like that. It doesn't matter if they're wrong - you can still challenge someone while being courteous (and not graphically detailing their mutilation). A huge part of your job as a PM is to be social, and this goes against every aspect of your gig.
Good PMs should be more Cicero, less Amy from Amy's Baking Company. Doing that requires a careful command of communication and responsibility, and I displayed none of these traits when I decided to go wild on this dude.
2.) There's a lot I didn't know about the account and the sales team's strategy such that my outburst could have ruined their plans a few months to a year or so out. Storage is a sector with a notoriously long sales cycle time, so the strategies often play out over a long period of time. It was extremely short sighted of me to do something like this.
3.) That meeting established a reputation that followed me for the rest of my days at NetApp. Because of my outburst, our execs (rightfully) sent me everywhere with senior staff and/or members of corporate counsel. I had to work hard to prove that an early twenty-something could be in charge of something as sensitive as security, and because of this incident I had to work doubly hard to prove that I wasn't a loose cannon.
Basically, I just got really lucky that I didn't get fired and cost the company a big customer.
Also, for the record, defeating industry-standard biometric scanners not as easy as dipping someone's thumb in warm water. It's obviously a lot more complicated than that, and the oversimplification was rhetorical in nature.
---
When I was 21 I almost lost several hundred million dollars by threatening to mutilate one of our customers.
In my senior year in college I worked full time as an intern PM at NetApp. I spent most of that time at work being groomed and prepared to be a full product manager, and given that my background was in cryptography I got pulled into a lot of customer meetings related to security.
One of our customers at the time was undergoing a big change with their security architecture, and I tagged along with one of the directors to the meeting. I was one of ten PMs giving talks on roadmap and our plans, and I had 30 minutes to convince their CIO and CEO that we could integrate our new systems well with the new security infrastructure they were rolling out.
It turned out though that the CIO and CEO weren't the only ones in the room. Joining them was the company's Chief Security Officer (CSO). Like me he was a young, rising star in their company with a lot to prove in a short period of time. He also didn't like me much from the get-go; when I walked in the room he sneered, and when I went to plug in my laptop to the projector he openly asked, "Is he really going to present alone?"
Most of my 5 slide presentation was instantly ripped apart. I had a good command of the tech involved, so the criticism wasn't on our findings. Instead, he nitpicked the design - the colors were off, the fonts weren't like the other presenters' (admittedly I did disregard the style designs, my bad), etc.
When I finished my slides and hit the time for questions, he laughed and shooed me away. "Good effort, but you clearly don't abide by our security practices." I stared at him with tired, dagger-piercing eyes across the podium. Not only did we abide by what they needed, but I'd spent all night working on this presentation (which combined with going to school full time meant that I was on very little sleep). I was pissed off, and I decided to push back.
Me: "Well what specifications are you referring to?"
Customer: "You don't understand. We are subject to a vast amount of compliance requirements inclu-"
Me: "-ding FIPS 140-2, PCI-DSS, FISMA..."
(I did my homework on the account)
Their CEO took notice at me pushing back and seemed to wake up from his "I don't care, when's lunch" stupor. As the CSO and I nerd battled like we were Sith and Jedi LARPers at Gen Con, a bunch of the account reps in the back of the room tried to get me to come off stage. My director let me stay.
Finally once I had proven that we fit the spec, the CSO changed tone to something ridiculous.
Customer: "Well what about biometric scanners? We need biometric scanners."
I blinked. Biometric fucking scanners? We're a storage company, not the Goddamn NSA. I responded that our authentication schemes supported most of the protocols that are used by bio-scanners, but he retorted that it needed to be first-party only.
I sighed, clearly exasperated, and responded bluntly. "We don't make biometric scanners. You don't need biometric scanners. They're expensive and none of your compliance requirements need them. It's complete overkill." The CSO immediately (and vehemently) shot back angrily, citing his military experience and how he was going to make an infrastructure that was "unhackable."
So I decided to turn the tables on him. "Okay, biometric scanners - what kind of biometric scanners do you need?" He gave me a basic list of specs, but having recently completed a homework assignment in my information security class (a class taught by a ex-NSA cryptanalyst who liked to talk about now-public faults in old security systems) on the topic I hit him back with the various faults in modern bio scanners - including the gory details on how you fraud them.
Me: "So you want bio scanners with feedback right? That's cool. Well what's to stop me from cutting off your thumb and swiping it like in the movies? Nothing. Unless of course you want to integrate temperature and humidity monitors, and even then I'll defeat it by running tubes into your cut-off thumb with warm water or soak it salt water in the microwave-"
At this point the CSO sat back horrified. The CEO of the company was dumbstruck, the account team in the back of the room was mortified, and my director was dying of laughter.
I proceeded to then go into detail about the faults of various retina scanners ("well I could pull out your eye and put a layer of Visine over the retina..") until finally the CSO sat back in his chair - both defeated in his designs and horrified at the glasses-wearing Asian kid in front of him that looked less like a brainy engineer and more like the Unabomber.
At this point the team decided to call it. I was quickly hurried out of the room - only after thanking the customers for their time of course - to where I met with the rest of the PMs in attendance who were literally doubled over with laughter. The account rep on the team later joined us and blasted me with a series of insults, noting that my insubordination might cost them the account and that I "clearly wasn't mature enough for my job."
I spent the next few days calling myself an idiot and getting ready to change my LinkedIn status. But when the email feedback report on my presentation came back I got the highest rating from the account's exec team. They noted that the CSO can "be difficult sometimes" and they appreciated that I "had a strong command and understanding of the security requirements of our space."
I spent the next few years at NetApp running product security. This event definitely came up during my year end review though, and since then I've become much better at presentation etiquette.
------
This is a screw up (note: original title was "fuck up") for a few reasons:
1.) You should never, NEVER, talk to customers like that. It doesn't matter if they're wrong - you can still challenge someone while being courteous (and not graphically detailing their mutilation). A huge part of your job as a PM is to be social, and this goes against every aspect of your gig.
Good PMs should be more Cicero, less Amy from Amy's Baking Company. Doing that requires a careful command of communication and responsibility, and I displayed none of these traits when I decided to go wild on this dude.
2.) There's a lot I didn't know about the account and the sales team's strategy such that my outburst could have ruined their plans a few months to a year or so out. Storage is a sector with a notoriously long sales cycle time, so the strategies often play out over a long period of time. It was extremely short sighted of me to do something like this.
3.) That meeting established a reputation that followed me for the rest of my days at NetApp. Because of my outburst, our execs (rightfully) sent me everywhere with senior staff and/or members of corporate counsel. I had to work hard to prove that an early twenty-something could be in charge of something as sensitive as security, and because of this incident I had to work doubly hard to prove that I wasn't a loose cannon.
Basically, I just got really lucky that I didn't get fired and cost the company a big customer.
Also, for the record, defeating industry-standard biometric scanners not as easy as dipping someone's thumb in warm water. It's obviously a lot more complicated than that, and the oversimplification was rhetorical in nature.
At the last company I worked for, we'd been trying to secure a corporate partnership with UPS for about a year. Everyone on the team had done backbreaking research, and the lead sales roles had spent several hundred hours crafting the higher levels of what this partnership would mean for both companies and drafting it into a beautiful (and I mean beautiful to read and to look at) partnership proposal.
And then I FedExed it to them.
We lost the partnership two to three business days later.
And then I FedExed it to them.
We lost the partnership two to three business days later.
Gil Silberman, Lawyer, technologist, social s... (more)
246 upvotes by Jennifer Newell, Samuel Henry, Diana Tan, (more)
In my first high school coding job — I guess that made me a tech intern — I burned down the entire office due to a cabling mistake. It was a total loss, displaced 100+ people, and destroyed lots of files, records, artwork, and a lot of the CEO's memorabilia. He never blamed me, but I think he knew it was my fault.
Certainly not the worst ever, but it was my third or fourth most costly professional mistake. After a couple decades in Silicon Valley, I think I've finally created more wealth than I've destroyed so I guess the world and I are even now.
Certainly not the worst ever, but it was my third or fourth most costly professional mistake. After a couple decades in Silicon Valley, I think I've finally created more wealth than I've destroyed so I guess the world and I are even now.
I was the CEO (which is short for doing everything you don't want to ask someone else to do) of a non profit network that did low cost and free internet and hosted many other non profit sites in a large metro area. The local PBS station was a large part of the PBS network backbone and had a server farm. They provided us with offices, space for our servers and other network services. I got keys to the front door.and pretty much every other door in the place. A huge... very huge... trust thing.
One Sunday afternoon I had to go do something to our servers... can't remember what. I took my son who was probably 4. When we got there he wanted to see Big Bird. I first started to explain to him... then decided it would do no harm for us to look around and see if we could find him. The building was uncharacteristically empty.
We couldn't find him so we went to the server room and I sat down at the console and started working. As always it was taking longer than I thought. I stopped paying attention to my son for a few minutes...
I realized suddenly that I was hearing the sound of fans winding down from the normally deafening roar.
My son had methodically started at one end of the room and began turning everything off.
OH MY GOD.... WHAT ARE YOU DOING!
Down went local. Down went national... down... down... down... I quickly started turning everything back on. Pretty soon realized that this wasn't going to get swept under the rug. While maybe everything would come back and reconnect... it would take time... it would be noticed by their admin who would be checking logs to see what the hell happened.
So I made the call. Their admin was already getting calls and looking at logs from home. I helped him get the network back up so he didn't need to come in.
When I finally got around to asking why he would do that... he explained that we were the only people in the building and nobody was using them. Mom had been complaining that Dad leaves his computers on even when not using them and it was a waste of electricity. Ya I have one in each room for convenience and assorted servers and test machines... they number somewhere in the teens.
One Sunday afternoon I had to go do something to our servers... can't remember what. I took my son who was probably 4. When we got there he wanted to see Big Bird. I first started to explain to him... then decided it would do no harm for us to look around and see if we could find him. The building was uncharacteristically empty.
We couldn't find him so we went to the server room and I sat down at the console and started working. As always it was taking longer than I thought. I stopped paying attention to my son for a few minutes...
I realized suddenly that I was hearing the sound of fans winding down from the normally deafening roar.
My son had methodically started at one end of the room and began turning everything off.
OH MY GOD.... WHAT ARE YOU DOING!
Down went local. Down went national... down... down... down... I quickly started turning everything back on. Pretty soon realized that this wasn't going to get swept under the rug. While maybe everything would come back and reconnect... it would take time... it would be noticed by their admin who would be checking logs to see what the hell happened.
So I made the call. Their admin was already getting calls and looking at logs from home. I helped him get the network back up so he didn't need to come in.
When I finally got around to asking why he would do that... he explained that we were the only people in the building and nobody was using them. Mom had been complaining that Dad leaves his computers on even when not using them and it was a waste of electricity. Ya I have one in each room for convenience and assorted servers and test machines... they number somewhere in the teens.
SlideShare down!
I was an intern at SlideShare . My project was to come with an 'admin panel'. The admin panel was essentially to serve as a one stop tool for creating, suspending, deleting, reconverting slideshows (and in some cases deleting the users). Most of the tech companies use a Master-Slave database. So all the reading operations are performed via the 'Slave' and the Master is supposed to handle all the 'write' operations. Master-Slave are in sync.
'Reading' operations can involve running a query such as to get the user's password, last login etc. 'Writing' operations would involve 'writing' the new password to the database, 'writing' the slideshow title uploaded and so on.
My code was supposed to 'read' all the user info. I had made a paginated interface limiting the user count on each page to be 50. The next 50 users should be fetched only if you navigate to the next page, else you can type the username in a search box to fetch that particular users' info. So in principle the query should be run on a 'slave'. Instead I ran this query, and yes I ran this query on the Master ( which is too busy doing the writing operations already!) :
SELECT * FROM users ;
I don't remember the exact user count then, but to add to this, I refreshed the page multiple times which used to perform this query.
SlideShare was a 503[Service Unavailable] for around 10 minutes.
Not a tech company. But this guy's story is a must read:
"Count to ten when a plane goes down…
Just a little under 31 years ago, I played a key role in a conspiracy theory that grew up around a passenger plane downed by a Russian missile. Trust me, I did not mean to be involved.
On September 1, 1983, Korean Airlines flight 007, a Boeing 747 with 269 passengers, was shot down over the Sea of Japan. At about 6am that morning, I arrived at my summer job at the American Embassy in Tokyo where my task was usually to start up the computer which had been turned off over night.
But on this morning, I realized the system was already engaged and that a surprisingly large number of workstations had been left on over night. While rare, I had seen this pattern before when a Washington deadline for information was looming.
Not long after I arrived in my office, I received a call from a secretary in the Agriculture Department who liked to play a computer game before her workday started. Her favorite game had a bug that regularly froze her workstation. This was the “bad old days” of computers and the only way to reset her station was from my central console.
On this day, I highlighted her workstation and hit the F6 key to reset. But my screen went temporarily black and then seemed to be starting again. I realized that I had mistakenly hit F7 and reset all the workstations in the embassy. This realization didn’t bother me much, because no one except the Agriculture section secretary was usually on the computer system this early in the morning.
But then all hell broke lose.
My boss, a Japanese computer engineer named Itoh, poked his head in the door. This was a shock because I had never seen Mr. Itoh before 10am ever. My job was to come in early and leave early and he arrived late and stayed late to shut down the system each night. He asked me what had happened. I told him I had shut down the system by mistake. He shook his head and ran down the hall.
Next, the head administrator, who I had only seen once in the computer room, walked in. He asked where Mr. Itoh was. I pointed down the hall. And he ran that direction as well.
More than an hour later, the Administrative Director returned to my office to explain what had happened. He told me about the Korean Airline disaster and that no one really knew what was going on, but that most of the information available was coming in from Japanese sources—first from Japanese fishing ships in the area and later from Japanese defense forces who were being dispatched to look for debris. A team of translators and US diplomats had been readying the first report for President Reagan at the time I turned off the computer systems. As this was a very early computer with limited backup capability, hours of work of dozens of experts had been lost when I inadvertently closed down the computer.
I, naturally, felt terrible and was, appropriately, fired.
It was only weeks later that I began to comprehend the effects of this single keystroke mistake. President Reagan was criticized in the press for his administration’s delayed announcement of the tragedy. But more troublesome, the reports that were being compiled in the US Embassy at the time of my error were meant to be shared with the South Korean government. As the team in Tokyo went back to rewriting the report—with clear evidence that the plane had been downed in the Sea of Japan—the South Korean government, working from flawed data, announced that the airliner had simply been forced to land in Russian territory and that all passengers and crew were safe.
That Korean announcement and the slow response by the US President—both caused by delayed real information—caused decades of conspiracy theories. Until the fall of the Soviet Union in the early 1990s, many Koreans clung to the hope that their loved ones were still alive and well in some Siberian prison camp.
So today, in the face of a Malaysian Airline crash in the Ukraine—and with all the associated speculation of 24-hour news organizations and the Tweetosphere, my advice is to take a deep breath, count to ten, and know that there is a very good chance that truth in the matter will be forthcoming very soon.
And let’s hope that there is no stupid 23-year-old with his finger on an important keyboard in this information chain."
Source: Count to ten when a plane goes down...
"Count to ten when a plane goes down…
On September 1, 1983, Korean Airlines flight 007, a Boeing 747 with 269 passengers, was shot down over the Sea of Japan. At about 6am that morning, I arrived at my summer job at the American Embassy in Tokyo where my task was usually to start up the computer which had been turned off over night.
But on this morning, I realized the system was already engaged and that a surprisingly large number of workstations had been left on over night. While rare, I had seen this pattern before when a Washington deadline for information was looming.
Not long after I arrived in my office, I received a call from a secretary in the Agriculture Department who liked to play a computer game before her workday started. Her favorite game had a bug that regularly froze her workstation. This was the “bad old days” of computers and the only way to reset her station was from my central console.
On this day, I highlighted her workstation and hit the F6 key to reset. But my screen went temporarily black and then seemed to be starting again. I realized that I had mistakenly hit F7 and reset all the workstations in the embassy. This realization didn’t bother me much, because no one except the Agriculture section secretary was usually on the computer system this early in the morning.
But then all hell broke lose.
My boss, a Japanese computer engineer named Itoh, poked his head in the door. This was a shock because I had never seen Mr. Itoh before 10am ever. My job was to come in early and leave early and he arrived late and stayed late to shut down the system each night. He asked me what had happened. I told him I had shut down the system by mistake. He shook his head and ran down the hall.
Next, the head administrator, who I had only seen once in the computer room, walked in. He asked where Mr. Itoh was. I pointed down the hall. And he ran that direction as well.
More than an hour later, the Administrative Director returned to my office to explain what had happened. He told me about the Korean Airline disaster and that no one really knew what was going on, but that most of the information available was coming in from Japanese sources—first from Japanese fishing ships in the area and later from Japanese defense forces who were being dispatched to look for debris. A team of translators and US diplomats had been readying the first report for President Reagan at the time I turned off the computer systems. As this was a very early computer with limited backup capability, hours of work of dozens of experts had been lost when I inadvertently closed down the computer.
I, naturally, felt terrible and was, appropriately, fired.
It was only weeks later that I began to comprehend the effects of this single keystroke mistake. President Reagan was criticized in the press for his administration’s delayed announcement of the tragedy. But more troublesome, the reports that were being compiled in the US Embassy at the time of my error were meant to be shared with the South Korean government. As the team in Tokyo went back to rewriting the report—with clear evidence that the plane had been downed in the Sea of Japan—the South Korean government, working from flawed data, announced that the airliner had simply been forced to land in Russian territory and that all passengers and crew were safe.
That Korean announcement and the slow response by the US President—both caused by delayed real information—caused decades of conspiracy theories. Until the fall of the Soviet Union in the early 1990s, many Koreans clung to the hope that their loved ones were still alive and well in some Siberian prison camp.
So today, in the face of a Malaysian Airline crash in the Ukraine—and with all the associated speculation of 24-hour news organizations and the Tweetosphere, my advice is to take a deep breath, count to ten, and know that there is a very good chance that truth in the matter will be forthcoming very soon.
And let’s hope that there is no stupid 23-year-old with his finger on an important keyboard in this information chain."
Source: Count to ten when a plane goes down...
Jason Perlow, Sr. Technology Editor, ZDNet. ... (more)
107 upvotes by Leonid S. Knyshov, Samuel Henry, Paul Mansfield, (more)
Back in the late 1980s I had a summer job as a repair technican for a local computer reseller for Altos. I was 17 years old. These were PC-like systems that had multiuser capabilities and used a form of UNIX called XENIX that was attached to green screen terminals made by WYSE. These machines could typically accomodate 8-12 terminals per small business.
One week I was sent in on a repair call to fix a broken terminal monitor. I had never repaired a monitor before. These were the days where you actually had to pull the monitor mainboard out, use hot glue on the connectors. Today you just swap entire screens out. But back then these things were expensive to replace so it was parts and labor.
I managed to get the mainboard out, and when I was connecting the power supply leads, which had to be hot glued on, that there were no markings for positive and negative terminals. The wires themselves were color coded but the power connections were not, I had forgotten which one was which. So I made my best guess. I connected the leads, and closed the monitor up.
This office was an accounting firm with about 20 people in it, and it was lunchtime, so a lot of stuff was going on. I plugged the terminal in and...
It made a huge electrical short sound, and sparks jumped out. Everyone started to look at me. "No problem guys, just a little short" I said. THEN THE THING CAUGHT FIRE.
One week I was sent in on a repair call to fix a broken terminal monitor. I had never repaired a monitor before. These were the days where you actually had to pull the monitor mainboard out, use hot glue on the connectors. Today you just swap entire screens out. But back then these things were expensive to replace so it was parts and labor.
I managed to get the mainboard out, and when I was connecting the power supply leads, which had to be hot glued on, that there were no markings for positive and negative terminals. The wires themselves were color coded but the power connections were not, I had forgotten which one was which. So I made my best guess. I connected the leads, and closed the monitor up.
This office was an accounting firm with about 20 people in it, and it was lunchtime, so a lot of stuff was going on. I plugged the terminal in and...
It made a huge electrical short sound, and sparks jumped out. Everyone started to look at me. "No problem guys, just a little short" I said. THEN THE THING CAUGHT FIRE.
I have to be anonymous.When I was interning at the Washington Post,I brought down the post's website for good 2 hours and publishing to the website for 5 hours.
While making a change in a very critical XML file,I forgot/was not aware that you have to escape the semicolon in the xml file.
However,my boss was cool and they didn't make a huge deal out of it.I eventually got hired at the Post also and gladly accepted the offer.
While making a change in a very critical XML file,I forgot/was not aware that you have to escape the semicolon in the xml file.
However,my boss was cool and they didn't make a huge deal out of it.I eventually got hired at the Post also and gladly accepted the offer.
This doesn't so much have to do with the job as it does with the people, but here goes...
I worked downtown in a high-rise as middle management for a tech-support outsourcer. I was the lead manager this night, and some of my employees had gotten hold of a couple of keychain laser pointers back when they first came out.
So they were shining the laser pointers out the windows of the 24th floor of this building, where our offices were located. And it just so happened that they shone them into the windows of the hotel across the street. And it just so happened that on that particular night, the President of the United States was staying on that floor of the hotel in the room that faced our building.
That's the only time I've ever met the Secret Service, and they don't have a sense of humor.
I worked downtown in a high-rise as middle management for a tech-support outsourcer. I was the lead manager this night, and some of my employees had gotten hold of a couple of keychain laser pointers back when they first came out.
So they were shining the laser pointers out the windows of the 24th floor of this building, where our offices were located. And it just so happened that they shone them into the windows of the hotel across the street. And it just so happened that on that particular night, the President of the United States was staying on that floor of the hotel in the room that faced our building.
That's the only time I've ever met the Secret Service, and they don't have a sense of humor.
I did a rm -rf * .txt ( yes, an accidental paste, with a spacing in between * and .txt)
I was copying from another file, and the command executed itself due to a new line at the end...
Boom. Gone.
I was copying from another file, and the command executed itself due to a new line at the end...
Boom. Gone.
tl;dr While Amazon intern, got bitched out by VP during one of the most internally politically controversial moments in company history for publicly lambasting the entire PR org
Amazon. Orwell's books were being deleted off people's kindles without their permission. Was causing PR havoc for the company. A lot of engineers were very idealistic, very critical of Amazon and the fact that it even had this capability, railing against the irony that copies of one of the most anti-authoritarion books ever written were being deleted in the same way it tried to warn against, etc. There turned out to be a giant thread on seattle-chatter@ heaping sarcastic remarks against the exec team (very rare) as well as the PR team. Since it was a giant sarcastic, idealistic cireclejerk, I decided to pitch in with a helpful and very common comment, calling the PR team "lazy" because they
"didn't carry pagers" and were probably "not working hard enough" at solving this problem.
I subsequently realized I had the misfortune of being the very last person to comment on this thread. Suddenly, my snark was the most visible comment in a very unAmazonian (read: free opinion sharing) thread.
Two minutes later I got a phone call..
"Is this? From that damn chatter thread?"
"Uhm.. yes.. who is this?"
"This is. Really? You really say shit like that? You really want to say shit like that? I'm the Vice President of Media Relations, it's seven PM, and I assure you, we are in fact working VERY HARD HERE. UNDERSTAND?"
"Uhmm, yes sir, I understand"
"KEEP THAT IN MIND NEXT TIME"
I immediately hung up, almost fell over from nervousness, and ran to a friendly engineering manager in the next cubicle and explained what happened. I was fucking terrified I was going to not get a return offer, get fired, disciplined, and otherwise blackballed. He helped me craft an apology email. That was it. I never heard from that VP ever again.
Edit 2: Amazon Erases Orwell Books From Kindle was the incident, not porn, not Wikileaks. I stayed away from the other two threads on s-c related to the above... because I had learned my lesson by that point...
Amazon. Orwell's books were being deleted off people's kindles without their permission. Was causing PR havoc for the company. A lot of engineers were very idealistic, very critical of Amazon and the fact that it even had this capability, railing against the irony that copies of one of the most anti-authoritarion books ever written were being deleted in the same way it tried to warn against, etc. There turned out to be a giant thread on seattle-chatter@ heaping sarcastic remarks against the exec team (very rare) as well as the PR team. Since it was a giant sarcastic, idealistic cireclejerk, I decided to pitch in with a helpful and very common comment, calling the PR team "lazy" because they
"didn't carry pagers" and were probably "not working hard enough" at solving this problem.
I subsequently realized I had the misfortune of being the very last person to comment on this thread. Suddenly, my snark was the most visible comment in a very unAmazonian (read: free opinion sharing) thread.
Two minutes later I got a phone call..
"Is this
"Uhm.. yes.. who is this?"
"This is
"Uhmm, yes sir, I understand"
"KEEP THAT IN MIND NEXT TIME"
I immediately hung up, almost fell over from nervousness, and ran to a friendly engineering manager in the next cubicle and explained what happened. I was fucking terrified I was going to not get a return offer, get fired, disciplined, and otherwise blackballed. He helped me craft an apology email. That was it. I never heard from that VP ever again.
Edit 2: Amazon Erases Orwell Books From Kindle was the incident, not porn, not Wikileaks. I stayed away from the other two threads on s-c related to the above... because I had learned my lesson by that point...
Amin Ariana, Technical Founder, hacker and ... (more)
29 upvotes by Chip Frank, Eamon Bohan, Ei Wai, (more)
A friend once refactored Enum values on a single line of code.
The company lost a million dollars by the next morning.
Turned out he had implicitly switched the definition of "on" and "off" at the ad campaign level, affecting billions of ad impressions.
He kept his job.
Engineering management got a major re-org for not having automated tests.
The company lost a million dollars by the next morning.
Turned out he had implicitly switched the definition of "on" and "off" at the ad campaign level, affecting billions of ad impressions.
He kept his job.
Engineering management got a major re-org for not having automated tests.
I was part of a team working on an interesting network game. We used to test our game against a super powerful server which was simultaneously being accessed by several other teams.All the final code was pushed to this machine. Seeing my pace of contribution and somewhat due to my due diligence, I got complete access to every machine including the mighty server. On one sunny night, when all my senses were at a high, I innocently ran
chmod -R 777 on root
and the rest is history...
For those who are wondering what chmod -R 777 root does, in simple terms it sets permission for every file on root to be read, write and executed. Linux considers this as a major security flaw(as it is designed on file permissions), and it becomes impossible for anyone to log in to the system. Since "root" is the boss of every other user, and its own castle has been compromised, so it doesn't allows anyone to come in, nor itself
chmod -R 777 on root
and the rest is history...
For those who are wondering what chmod -R 777 root does, in simple terms it sets permission for every file on root to be read, write and executed. Linux considers this as a major security flaw(as it is designed on file permissions), and it becomes impossible for anyone to log in to the system. Since "root" is the boss of every other user, and its own castle has been compromised, so it doesn't allows anyone to come in, nor itself
Jeff Kesselman, 25 years in the video game ind... (more)
36 upvotes by Clarence Leung, Alex Giannakakos, Bulat Bochkariov, (more)
So, this isn't an intern story but an entry level employee story, but Jay's made me think of it.
I was an early employee at a later to be fairly famous game company. We were making the earliest CD-ROM games for the "next gen" platforms which were going to be CD-rOM based, if that gives you an idea of the timetable.
Anyway, we had a junior game designer with a bit of an attitude. Our junior game designers also were the level creators. He lost an argument with more senior heads on a design feature.
Not satisfied to let it be, he buried a rant about "how much better this game would have been if they had listened to me" in one of the levels. Back in those days you had to "freeze the bits" to create the "gold master" in order to produce a CD-rOM product. The gold master was literally a CD made from solid gold that was used to stamp out the plastic CDs. A gold master cost $5,000 to $10,000 to make. needless to say, it was a disaster if you had to make more than one.
The hidden rant was caught JUST before we went gold, thanks to the eagle eyes of one of the game testers.
In 25 years in the industry, that is the ONLY time I have seen someone fired with a security guard in tow. That guard watched him while he cleaned out his desk, took his keys and physically escorted him out of the building.
The game industry is a small industry. I would be surprised if he ever worked in it again.
I was an early employee at a later to be fairly famous game company. We were making the earliest CD-ROM games for the "next gen" platforms which were going to be CD-rOM based, if that gives you an idea of the timetable.
Anyway, we had a junior game designer with a bit of an attitude. Our junior game designers also were the level creators. He lost an argument with more senior heads on a design feature.
Not satisfied to let it be, he buried a rant about "how much better this game would have been if they had listened to me" in one of the levels. Back in those days you had to "freeze the bits" to create the "gold master" in order to produce a CD-rOM product. The gold master was literally a CD made from solid gold that was used to stamp out the plastic CDs. A gold master cost $5,000 to $10,000 to make. needless to say, it was a disaster if you had to make more than one.
The hidden rant was caught JUST before we went gold, thanks to the eagle eyes of one of the game testers.
In 25 years in the industry, that is the ONLY time I have seen someone fired with a security guard in tow. That guard watched him while he cleaned out his desk, took his keys and physically escorted him out of the building.
The game industry is a small industry. I would be surprised if he ever worked in it again.
I got a CEO to retire. During one of my internships, I asked the CEO in a open-to-all meeting if he's doing his best to keep the company aligned with its mission and vision. He asked me who I was, thought for a few minutes [with about 200 people in the audience], walked out of the room, took a few more minutes, walked back in, and announced retirement.
More details about answer, since a lot of comments seem to be the same question:
+ This was not a mega mega company. It was a 500 people company.
+ The CEO was, afaik, not planning to retire. At the same time, there were already talks about the CEO being a possible candidate for getting fired soon as the company wasn't doing too well.
+ The CEO seemed to be in a state of slight depression; frankly, in his speech, he said that all he needed was someone to tell him where he lost it. And a question which most people did not ask [for not being on the CEO's let-go list], was asked by me.
More details about answer, since a lot of comments seem to be the same question:
+ This was not a mega mega company. It was a 500 people company.
+ The CEO was, afaik, not planning to retire. At the same time, there were already talks about the CEO being a possible candidate for getting fired soon as the company wasn't doing too well.
+ The CEO seemed to be in a state of slight depression; frankly, in his speech, he said that all he needed was someone to tell him where he lost it. And a question which most people did not ask [for not being on the CEO's let-go list], was asked by me.
Scene: Summer '05 me as a software engineering intern in the IT department of Intel (we were building internal tools for IT).
First day my internship supervisor:
Supervisor: "Do you know perl?"
Me: "No"
Supervisor: "Learn perl and edit this script that manages a dhcpd.conf file for production to check for big file size changes and reject a change if it's by more than +- 50%."
One week later I'm finished. Write something to my Supervisor who's out at the Guadalajara, Mexico office that week. He rolls it out without checking on my work (it actually worked and was to spec, it was just too literal).
Later that day I get pulled aside by his supervisor: "How did you roll out a script that took down all of fab production after just 1 week?" (to be clear, nobody blamed me, even my supervisor was okay, but they did make a major change to release protocols from that point forward.
P.S. The problem was that they would wipe and reload the dhcpd.conf file at some point and basically this script prevented it form going above size 0 at that point causing everybody to not be able to get an IP.
First day my internship supervisor:
Supervisor: "Do you know perl?"
Me: "No"
Supervisor: "Learn perl and edit this script that manages a dhcpd.conf file for production to check for big file size changes and reject a change if it's by more than +- 50%."
One week later I'm finished. Write something to my Supervisor who's out at the Guadalajara, Mexico office that week. He rolls it out without checking on my work (it actually worked and was to spec, it was just too literal).
Later that day I get pulled aside by his supervisor: "How did you roll out a script that took down all of fab production after just 1 week?" (to be clear, nobody blamed me, even my supervisor was okay, but they did make a major change to release protocols from that point forward.
P.S. The problem was that they would wipe and reload the dhcpd.conf file at some point and basically this script prevented it form going above size 0 at that point causing everybody to not be able to get an IP.
When I was interning at Baylor College I set the entire Human Genome Project back a week. I was writing my first program to make use of a grid to align human and mouse genomes. I was sending large amounts of data around the network. A bug in my code caused my program to fork infinitely and eventually burnt out the controller running the network array. It took down the college's connection
(this isn't Reddit) to the internet for two days. When we got it back I did it again and that was the last time they let me use the grid......
(this isn't Reddit) to the internet for two days. When we got it back I did it again and that was the last time they let me use the grid......
I got the IP of my company blocked by Google, such that, each person had to enter a CAPTCHA to get back their access to Google.
I had written a scraping script that parsed the Google search page results and filled our database with retrieved values. But, alas, I forgot that the computers were very fast. So, supposedly, my script was making approx 5-10k searches each second.
I tested the script, put it on one of our servers, ran it in the background and left for home. GOOD, no?
NO!!
After an hour or so, Google had blocked our IP. And next day, each employee was searching for the cause of the CAPTCHA.
Note : It happened some 2 years ago, so the values might not be exact..
Update: It seems I forgot to tell that the CEO paid me a visit that day, for the first time.. *poker face*
I had written a scraping script that parsed the Google search page results and filled our database with retrieved values. But, alas, I forgot that the computers were very fast. So, supposedly, my script was making approx 5-10k searches each second.
I tested the script, put it on one of our servers, ran it in the background and left for home. GOOD, no?
NO!!
After an hour or so, Google had blocked our IP. And next day, each employee was searching for the cause of the CAPTCHA.
Note : It happened some 2 years ago, so the values might not be exact..
Update: It seems I forgot to tell that the CEO paid me a visit that day, for the first time.. *poker face*
No comments:
Post a Comment