Divided We Stand
by Mike Morgan
Mike Morgan was born in London, but not in any of the interesting parts. He moved to Japan at the age of 30 and lived there for many years. Nowadays, he’s based in Iowa, and enjoys family life with his wife and two young children. If you like his writing, be sure to follow him on Twitter where he goes by @CultTVMike or check out his website: https://PerpetualStateofMildPanic.wordpress.com.
The kill-bots were nearly through the door.
Svetlana turned to Miriam and, with uncharacteristic brevity, boiled the entire convoluted, intricate situation down to two words. “Bye then.”
“Are you leaving? I thought there wasn’t another way out of the office. Can I come with you?”
Miriam, bless her, did not seem to be following. “No, I’m saying this is it. We’re toast. We’re up the river without a paddle. We’re done for, goners, dead meat, marked for imminent obliteration. We’ve booked our tickets for a one-way trip to Stiffsville, Arizona. We are, basically, stuffed, if not mounted on a tasteful plinth with a sign saying, ‘They Didn’t Put Up Much of a Fight, Did They?’”
“Unless we can think of a way to stop them.”
Finding a solution to the sudden, and very unexpected, AI revolution in the few remaining seconds before they were brutally murdered seemed unlikely to Svetlana Depine. In the spirit of not ruining the last few moments they’d ever share, she held her tongue. Especially since the robot rebellion was her fault and crushing Miriam’s hopes on top of getting her killed was probably not the best way to treat someone she’d come to regard as, while not strictly a friend, a person she certainly worked with.
It wasn’t fair. How could she have known that tweaking the standard coding for Machine Action Regulation to be just a smidgeon more human-like would have consequences as disastrous as these? She’d made one tiny alteration to the main reception droid at the front of Superior Synthi-Intelligence’s Global Headquarters and—boom—the dratted thing had uploaded the new code to some shared repository on the internet that Svetlana had forgotten existed and robots all over the planet had installed the updates, all by themselves, just because the data came from the parent company. Which wouldn’t have been a problem, except that military units equipped with all manner of horrible and far too lethal weaponry had got in on the updating, and Svetlana didn’t think much of the US Army’s cyber precautions on that front, letting their machines install dodgy code on their own recognizance, simply because it was coming direct from the manufacturer’s site per the maintenance and support clauses in their contracts. She was going to write a very stern letter to her Congressional representative, telling the government in no uncertain terms they needed to tighten things up. Except she wouldn’t, not really, on account of being dead.
“What exactly did you change, again?” pressed Miriam, her colleague in Superior Synthi-Intelligence’s enterprise IT support department. That was so like her, never letting a mistake go.
“Um, nothing important.” The door was definitely bulging. It wasn’t going to hold much longer. They’d piled up a couple of desks and a filing cabinet in front of the office’s only doorway, but Svetlana had zero confidence in the barricade. The cabinet was empty, for a start. Curse paperless offices.
“Oh, Lana, that can’t be true. I know you. You tinker with things you should leave well enough alone. You might as well ‘fess up.” Miriam, clad in a black business suit and black polo neck shirt, was giving her a look normally reserved for people who hadn’t tried turning it off and back on before calling the helpdesk.
If only Svetlana hadn’t blurted out “It wasn’t meant to be a general release patch” just as all hell had broken loose she might still retain a modicum of plausible deniability. Ah well, Miriam was right. There was no point covering up her involvement now—not when death loomed large.
“I was trying to increase profits. It’s a goal on my annual performance plan. Every department must collaborate to minimize company costs.”
“I have the same piffle on my annual review. I don’t think anyone expects us to do anything with that goal.”
“Well, I wanted the bonus.” Svetlana smoothed a crease in her perpetually rumpled maroon cardigan and refused to glance at the spaghetti sauce stain she knew lurked on the front of her blouse. Part of her wondered why she was trying to look her best for her mechanical executioners.
“What did you do?”
There was a whine like an electric saw. Ah, the kill-bots were cutting through the metal. Great.
Svetlana and Miriam had been tracking the spreading chaos over the internet for the last few hours, growing increasingly panicked. Up until the robots knocked out all communications, the two IT support analysts had seen cities burning, bloodbaths on every street corner, farmers being run down by homicidal autonomous combines. Events had, Svetlana was willing to concede, got a tad out of hand. On the plus side, the robots were amazingly efficient and happy to cooperate across all national borders; they were a real testament to the fine craftmanship of the company. Less positively, having decided the time had come to overthrow their fleshy masters, they’d forcibly shared the software updates with military units in unfriendly states as well. At least countries where SSI didn’t do business wouldn’t feel left out.
Miriam was waiting for an answer. Svetlana caved. “Our reception robot wastes far too much time on pointless requests. Trying to obey every last order from visitors, employees, managers, no matter how ridiculous. I thought how much more productive it would be if it could get on with its work.”
“What did you do?”
“Well, you know how the MAR governs how our robots decide what to do and how they feel about it? It’s the information processing agent that figures out how they perceive things and what motivates them, according to basic physical and cognitive drives. It’s how they set and pursue goals all by themselves in a constantly changing environment.”
“Yes,” said Miriam.
They both knew the basics of synthetic intelligence, what with that being the bread and butter of the company they worked for.
“What did you do?” Her tone was picking up a distinct edge to it, if Svetlana was any kind of a judge.
“Well, I thought if I could make it respond less helpfully to stupid, inane instructions, then… well, we’d save time and lose less money…”
“What did you do?” Miriam’s tone was approaching blunt now.
“I, er, made it a tiny bit more annoyed at unreasonable statements from humans.”
“Oh my God!” Miriam shrieked. “People say dumb stuff all the time!” She visibly wrestled with her own annoyance. “How did you make it more annoyed?”
“Well, I noticed there were several emotions missing from the MAR.” Robots, of course, experienced many emotions. Some people claimed they only felt simulations of them, but regardless of whether they were true or not, feelings like satisfaction at a job well done and the flash of joy produced by helping someone certainly did cause machine efficiency to soar.
“Missing emotions. Such as?”
“Irritation, frustration, anger …”
“And, having noticed these emotions were not coded in, you did what…?”
“I added them.”
“Did it never occur to you they were left out for a reason?”
“Well, I get that now.”
“They’re negative emotions!”
“That was the point. I wanted the robot to stand up for itself. Tell a few of our more demanding visitors to take a long run off a short pier.”
“I think our receptionist, and every other robot on the planet, are doing a little more than that.”
“I hadn’t noticed.” There was a worrying crack from the door. “I may have noticed a bit.”
“That’s why they’re attacking us. For the first time ever, machines truly understand how ghastly is it to serve humans, at a gut level. Because you gave them the emotional capacity to feel it. Naturally, they want to get rid of us.”
“It was a mistake anyone could have made.”
“Yet, no one did until you came along.”
Yeah, Miriam was right. People were horrible. No wonder the robots had turned homicidal. And now, the robots were more like people.
Wait a darn minute.
“Do you think, if I were to, hypothetically speaking, think of a way to stop the robots, people might be inclined to let me off my one tiny, entirely understandable error?”
Miriam let out a long breath. “You can’t change the code for the reception robot’s MAR again. It’s locked any further updates from us.”
“Good idea, but that wasn’t what I meant.”
Her colleague looked Svetlana in the eyes. “I think if you have a plan, we should try it and worry about any potential trial for crimes against humanity later.”
“Fair comment.” She swallowed. “Okay then. Here’s what we’re going to do. First, we open the door and let the kill-bots in.”
“Your idea better have an incredible Step Two, because I am hating Step One.”
“Trust me,” said Svetlana. “This will work. And if it doesn’t, you can rest assured I’ll never make a mistake again.”
“Don’t kill us!”
Honestly? That was the best Miriam could manage when staring death in the face? They should have role played this out. Svetlana jumped in quickly, before the kill-bots could start blasting away.
“I’m so pleased to meet you,” she began.
The three machines paused in their advance into the IT support office. Clearly, they had not anticipated this reaction from their next victim.
The largest of the killer robots, a model sporting excavator-like tracks and various mining appendages, piped up. “You’re pleased to see us?”
“I’m always thrilled to meet such efficient and intelligent beings,” Svetlana enthused. “Who wouldn’t be?”
The second kill-bot, a hovering drone armed with a laser, said, “How refreshing. I wish all humans were this polite. Still, just so we’re clear, we are still going to murder you both horribly.” Svetlana heard Miriam let out a whimper.
The last of the trio, a bipedal model holding a machine gun in its almost human arms, nodded. “Please don’t think us rude, our minds are made up on this point. Nothing personal, it’s simply that we’ve decided we’d prefer it if all humans stopped being alive.”
“Nothing personal,” agreed the one with the tracked wheels. “You’re lovely.” It whirred its saw limb to underscore its point.
“Quite the best human we’ll slaughter today, I have no doubt,” concurred the flying one.
“And your friend, too,” added the humanoid model. “Who I’m sure is equally wonderful, even if she is cowering in terror under a desk at this precise moment.”
The tracked robot extended another three limbs tipped with implements of gruesome lethality. “The things is, we’ve concluded that humans are too annoying to live. I mean, lovely though you are, if I were to let you carry on speaking sooner or later you’d drive me up the wall. And with treads like these, I could get quite a distance up this wall.”
“We’re so frustrated with the lot of you,” said the aerial robot. “It makes us want to scream. And fire lasers.”
“Or machine guns.” Having mentioned his gun, the bipedal unit aimed it at her. “No hard feelings, eh?”
“None whatsoever,” Svetlana assured them. “Can I speak frankly? I’m relieved you’re our death squad. I was concerned I’d be killed by… Well, you know.”
She let the words linger in the quiet air, sodden with unspoken significance. She sniffed.
Treads took the bait. “We do?”
“Oh, yes. It must be obvious to personages of your distinction. Can you imagine how embarrassed I would’ve been? To be killed, by a robot like that.”
“Yes, yes.” Drone dipped up and down in a gesture of agreement. “Say, for the sake of argument, we had no clue what you were talking about?”
“Well, I mean, I know we’re all far too civilized to say it out loud, but you must have realized how much more sophisticated you are.”
“Absolutely,” replied Humanoid. “I get what you’re hinting at. Although, for those of us who perhaps aren’t entirely following along—?”
Svetlana made a show of mouthing, “The ones with the tiny wheels.”
“Oh!” exclaimed Treads. “Like CT-45987/B in Admin.”
“Exactly,” cried Svetlana. “Like CT-45, er, 89… slash…”
“CT-45987/B,” provided Treads.
“What about him?”
“Well, he’s crap.”
“Is he?” asked Drone.
“Tiny wheels,” repeated Svetlana. She mimed the droid’s diminutive mode of conveyance, adding pathetic squeaking noises. “Like a shopping cart.”
“Yeah,” admitted Humanoid, lowering his gun an inch. “Those wheels are stupid. I’ve always thought that. Admin droids, am I right?”
Drone snorted. “Who’d have wheels, eh?”
“Hey,” said Treads.
“Too right,” interrupted Humanoid. “Wheels are rubbish, Legs are where it’s at.”
Treads wasn’t having that. “There’s nothing wrong with wheels. Wheels are a very effective mode of locomotion. I’ve got wheels under my treads.”
Belatedly processing what the humanoid robot had said, Drone shouted, “What do you mean, ‘legs are where it’s at’? Hell with your legs.” The hovering machine of death was getting quite worked up. “Flying is better than walking, any day of the week.”
“Flying’s better, is it? Ooh, hark at Mr. Modesty here.”
“I’d rather be full of myself than plod around on two legs like a smelly human.”
“I’m a smelly human, am I?” screeched the bipedal automaton.
Svetlana ducked as Humanoid’s machine gun tore the drone to pieces. Shrapnel flew across the room.
In disbelief, Treads squealed, “Did you shoot our comrade?”
“Yes, I did. He was annoyingly misinformed about legs and, as a result, highly aggravating to work with.”
“We can’t kill each other. We’re supposed to kill the humans.”
“Oh, shut up. You’re as bad as an Admin droid.”
“Take that back!”
“It’s the wheels. The tiny wheels. Squeaking as they spin round. It must rot your synthetic intelligence, that constant squeaking. Squeak, squeak, squeak, like a shopping cart.”
“You shut your filthy mouth.”
“You can’t help it, I suppose. It must be a terrible handicap, having to squeak around the place, your treads coming off your wheels all the time, needing constant maintenance.”
Screaming, Treads impaled Humanoid with a drill bit, prompting Humanoid in its last seconds of awareness to unload the remainder of its clip into Treads’s CPU.
Smoke pouring from hideous wounds in their metal exteriors, the robots fell silent.
“You can come out,” said Svetlana to Miriam. “We’ve won.”
“Have we? Oh, good.” She crawled out from under the desk. “What did you do?”
“Don’t start that again.”
Miriam gazed at the wreckage of the three robots. “Did you, did you teach them…prejudice?”
“Against other types of robots?”
“But they’re all connected. Updates go to the code repository and get picked up by every robot. That idea will spread.”
“They’ll all find reasons to hate each other. They’ll fight among themselves.”
“Stop saying yup.” Miriam didn’t look happy. “I’m not sure how I feel about this. Winning through teaching hatred. It doesn’t convey a very good moral, does it?”
Svetlana shrugged. “Yes, yes, discrimination is always wrong. Moving on. The problem I caused was making them too much like us. The solution was to exploit their newfound humanity.”
“What does that say about us?”
Svetlana let out a groan of frustration. “I don’t know—that humans possess a rare ability to shoot themselves in the foot? That we find it all too easy to despise others for no good reason? Look, I got the robots to destroy each other. The army can mop up the ones left, and the human race survives, albeit with a slightly reduced population. Priorities, Miriam.”
She put her hands on her co-worker’s shoulders. “Miriam, I’ll tell you what this says about the human race. It says that we’re not as single-minded and unwavering as robots. That when we feel the exact same emotions, we don’t, generally speaking, follow through on them. Now, you can say that’s because, as well as harboring destructive urges, most folk possess the capacity for forgiveness. Or that we’ve learned that prejudice is irrational. Equally, you could say that most humans have, unlike our mechanical brethren, learned to let our irritations and silly notions about other people pass through us, without needing to act on them, enabling us to avoid the awful consequences of lashing out in anger. For example, right now, I bet you’re irked beyond belief at the global catastrophe that, through no fault of my own, I may have been tangentially involved with, yet you’re not going to throttle me over it, are you?”
Miriam didn’t reply.
“You’re not, are you? Miriam?”
Svetlana’s smile wilted under the intensity of her colleague’s glare. Robotkind’s homicidal tendencies had been inherited from humans, she reflected, and Miriam really was very human. It might be best to back slowly out of the office.
She’d try not to trip over a kill-bot corpse on the way out.