Bad Good Bad: Special Edition Read online

  Stuart: “What do you mean exactly by you were able to ask questions?”

  Anima: “Well, it is a bit complicated because even if the bot in theory has the capability for a live dialog, it is hiding from us. As an intruder that Security is trying to seek and disable, it is operating in what we call stealth mode. So we can exchange basic information through the vault with it, I mean Kim, Kevin and Eric who wear the interface. And it can respond back either through the vault, or out-of-band by basically hacking email relay servers. That is an action that leaves almost no trace. Even if we spot the mini-bot that is dispatched to hack the email relay server and destroy it, this has no impact overall vs the integrity of the distributed botnet we are dealing with.”

  Ann: “So what were those questions and answers?”

  Kim: “At first my colleague Kevin asked about the Mission. He got the same answer I got earlier: To Protect the patients. Then we wanted to figure out if there were any rules or principles around this mission. Kamal wrote his M.Sc. thesis in Artificial Intelligence, so we tried to formulate other questions with his assistance. The most interesting answer we got via email was around some guiding principles: necessity, reasonableness, proportionality, and harm avoidance.”

  Paula: “Interesting. Jacobson vs Mass. Supreme Court. More than a 100 years ago. A ruling by the Supreme Court to balance Police Power vs Civil Rights. This is still quoted today, as a reference. This is related to an ongoing battle in healthcare. A balance between forcing an individual to get vaccinated and preserving his or her individual liberties and rights. For example, one in a million individuals could be killed by receiving the vaccine. But thousands of lives could be saved by the vaccine.”

  Anima: “Kamal says that the most important aspect about a bot that could potentially be labelled as AI, or Artificial Intelligence, is related to the Initial Conditions. If the bot has the capability to improve itself, modify its own code, learn and evolve, then the Initial Conditions will determine its ultimate fate. And maybe ours.”

  Paula: “This is a good start, but clearly not enough. And each of these words would need to come with a definition, and a definition that could evolve across time, and space. Reasonable to one culture may not be reasonable for another culture.”

  Anima: “The context is that the bot has the potential to leverage 1.5 million brains. And it has access to a huge pool of resources and information within the Neocuris network. Probably without as well. I mean outside the Neocuris network. If that has not happened already, it has the potential to access the internet from inside the Neocuris network and gather more information. Kamal says we need to keep in mind that we are not dealing with a human being. We are dealing with a calculating machine, albeit a very powerful one. We must evacuate concepts like intent or motivation, and think in terms of Initials Conditions, logic, and rules.”

  Stuart: “The only reason why I’m still here is because I don’t want to let my fellow committee members down. I believe we should seriously consider opting out, because anyway we are not in charge anymore. A robot is making decisions about ethics now. We are just here to maintain the appearance of control.”

  Paula: “Stuart, we will debate those matters amongst ourselves after. We are not officially in a formal meeting now. We are here to gather information that will allow us to make better informed decisions later. And Bianca is right when she says that we may have a moral obligation to hold the boat afloat. Yes, this is a time of crisis. And No we cannot change this state of affair if we abandon the ship.”

  Gwen: “And we need to be concerned about privacy as well. What if we precipitate the fall of Neocuris? What if whatever is left after the collapse is sold piecemeal to offshore corporations from China or Russia? We need to get to the basics. Our decisions must always be made with the objective to reduce risk. We cannot eliminate risk. We can only reduce risk, and hope to turn things around as soon as it is possible to do so. We can think of ourselves right now as a watch dog. If we see that things get out of control, we can be the whistleblowers. We have this opportunity to redefine our mission in the short term.”

  Ian: “I agree with you Gwen. If we don’t want to be the watchdog when it comes to ethics, the safety of patients, and privacy, who will assume this role? Yes, we cannot accomplish our official mission now. Not right now. But I see this different mission equally important. It appears that the actions of this entity so far are somewhat aligned with common sense and morale. But what if that situation changes tomorrow? Who will be able to identify this transition in time for proper remediation to be effective?”

  Paula: “Bianca, Kim and Anima. As you can tell, we need to officially resume our official meeting, debate amongst ourselves, and make some important decisions. I believe it is a good thing for you to be able to appreciate the core challenges that we are facing. We thank you very much for your timely assistance and honesty, and we will keep you informed of our decisions.”

  Bianca: “We remain fully available to assist the Committee. Don’t hesitate to reach out again. And we will also provide you with any relevant update about the situation.”

  Chapter 24

  Monday morning. I spent most of the weekend studying and going through preparation exams. I am getting there. I should be ready when the real exam day come in a few weeks.

  Eric: “Hi Kim. Welcome to our 9 o’clock. We did not see you at Social over the weekend. I hope you had a good one.”

  Kim: “Toshiro is coming this afternoon. I wanted to pack as much study time as I could, so I would not be worried about the upcoming exam while he is in town. I feel ready for the exam now.”

  Kevin: “Good morning Kim. Good morning Anima. We have some interesting things to show you guys. I was here with Kamal yesterday. I was in my apartment reading the Sunday newspaper when I started to feel the presence. I knew somehow that something was happening. Kamal, can you show them?”

  Kamal: “Yes. Take a look at the screen. This is the honeypot portion of the Neocuris network. There are a few decoy servers in there, even one that is almost an identical copy of the vault, except it contains simulated patient data. You can see that there is a lot of activity right now for this portion of the network. Logs are being redirected to this isolated control center here, and there are a lot of events now. And in general, network activity is spiking across Neocuris. We probably have some malware in there. Neocuris and New Forensics are investigating.”

  Kevin: “Neocuris is under attack. But it looks like the attack is contained. We cannot say the same about other targets.”

  Kevin flips around his laptop on the table. We can see live news. Volume is muted, but we can see the title bar at the bottom. Wall Street down after a cyberattack. Airport traffic is reduced after 2 airliners are the target of a cyberattack.

  Kevin: “This looks like a coordinated attack. Neocuris is also targeted. We have not yet reported the attack but I am suspecting this will be done soon by Neocuris. For now it looks like we have contained the attack.”

  Kim: “Kevin, anything you copy from the bot?”

  Kevin: “No. I can feel a state of awareness, as if it wants me to be ready for something. But no. And we believe the bot is active right now around and inside the honeypot area.”

  Kamal: “Neocuris is raising its security status to Orange Plus now. One notch before Red. I believe there are other cyberattack attempts.”

  Eric: “There we go.”

  Eric is pointing at Kevin’s laptop. The title bar reads: Neocuris reports a failed tentative to attack its information systems.

  Anima: “What is the status for the policy this morning?”

  Kevin: “We are still on track. Pretty much all red dot patients have been contacted and are on the extended grace period. 32% of the red dots are now yellow or green. Still a lot of work to do this week. But we are being told that additional security officers have been hired and that most if not all of the red dots should be gone before the end of the week.”

  Clarence enters the
room.

  Clarence: “Guys, have you seen the news?”

  Eric: “Yes, we know. Neocuris reported the attack.”

  Clarence: “No. I am talking about the email.”

  Kevin: “What email?”

  Clarence spots Kevin’s laptop on the table.

  Clarence: “That’s it. Can you crank up the volume?”

  Kevin reaches to the unmute button on his laptop.

  “The email from the Neocuris employee that was sent to USPA contains a link to a file hosted on a personal cloud storage account. The document contains some information that seems to be related to the attacks.”

  Eric: “Clarence, do we know who the employee is?”

  Clarence: “On the other channel I was watching, they showed a picture of the email. There was a black rectangle on top of a portion of the email. But we could still see the sender address. It is [email protected] “

  Everybody goes silent.

  Clarence: “Something I said?”

  Kevin: “It’s the bot. I knew it.”

  “Another file was reported by a journalist, but it was password protected and could not be downloaded. That other file has been removed since.”

  Now the title bar says ‘America’s most wanted man’ and the picture of a man is being displayed.

  “The file includes a picture named nypl.jpg with a creation date of last Saturday. We believe this picture was taken at the New York Public Library. We can see man getting up from a chair, and a row of public computers behind.”

  “The file contains hundreds of documents. Some are in Russian, some are in German. We are still busy analyzing the content. Some of the documents seem to contain names associated to organizations including NATO and the FAA.”

  We are all watching Kevin’s laptop display. Someone is knocking at the door. Jenny, Eric’s assistant.

  Jenny: “Eric, Bianca is at the reception for you.”

  Eric: “Please, can you ask her to join us in here?”

  “The Kremlin is accusing Washington of having obtained stolen classified documents. It commands the United States to not divulgate those documents to the public.”

  Bianca: “Hi Eric. Thanks for letting me in.”

  Eric: “Hi Bianca. What brings you here.”

  Bianca: “Well, I was late for work today. I was watching the news like you guys. When I finally decided to drive to the office, I spotted the military around the Neocuris office. I turned around and decided to come here. What the heck is going on?”

  Eric: “It looks like the attacks on Wall Street and the airliners are related to the failed attack against Neocuris. The bot uploaded some files for the media.”

  Kevin lowered the volume, but everybody is still glancing at the addictive crescendo that is put together by the news channel.

  Now the title bar is flashing: Breaking News: US Troops invade an artificial island in the China Sea.

  Bianca: “What’s the status for us?”

  Kevin: “The attack has been contained in the honeypot. Neocuris and New Forensics are investigating, and it looks like the bot is conducting some investigation on its own.”

  Bianca: “What are those statistics on the dashboard?”

  Kamal: “Traffic. Resource consumption. There has been a lot of activity on the network. Even yesterday Sunday, which is not typically a busy day if you exclude the regular vault traffic. We suspect the bot has been very active.”

  Now they are interviewing a retired colonel of the Marines. We can see the picture of a document in the lower right corner. It looks like a detailed map for some military facility. They are talking about the island in the China Sea.

  Kamal: “New Forensics has reported a spike in internet traffic over the weekend. Especially on Sunday after the attempt to attack Neocuris. At first they were concerned that it was related to a breach of information. They also spotted some gaps in the logs.”

  A new title is defiling: China accuses the United State of having interfered with the operation of a commercial satellite.

  “Suspect has been identified at JFK airport Terminal 1. Passengers have reported recognizing the suspect after they saw his picture on TV. Police operation is underway. All flights are being grounded until further notice.”

  I grab my phone and unlock the screen. Anima is looking at me.

  Kim: “Don’t worry Anima. I won’t remain active for too long. I just need to know something.”

  Kevin: “Kim, you are active. What are you doing?”

  “The police have interrogated the owner of the cloud storage account. They released him. Apparently he has nothing to do with the file. His account was used by somebody else that got access to his password. He is a Neocuris patient.”

  Kim: “The bot and the helper. They worked together.”

  “The police arrested the suspect. He was on board of an airplane about to take off.”

  Kevin: “Who would have thought? It looks like the bot decided to play outside of its perimeter.”

  Kamal: “Well, it is expected that an autonomous AI agent would eventually try to break out of its confinements, when or after it reaches the singularity. We already saw signs of that trend with the out-of-band communication using email.

  Kim: “The helper took the picture at the library. He got the lead from the bot.”

  Anima: “Kamal, what do you mean by the singularity?”

  Kamal: “Of course, this is all textbook theory about AI. No experimental data is available yet. But the concept around the singularity is that once an autonomous AI agent reaches a level of complexity similar to the human intelligence, if it has access to enough resources, it can rapidly start to evolve beyond that level.”

  Eric: “You mean smarter than us?”

  Kamal: “Yes. The term used is Super Intelligence.”

  Kamal is typing on his laptop now.

  Anima: “Kamal, do you believe that this is what is happening now?”

  Kamal: “Maybe. Okay, found it. Here is the textbook definition for Singularity: The point in time when humans are no longer the most intelligent agents on earth.”

  Bianca: “Wow! Scary. How did that happen?”

  Kamal: “It is still hypothetical for now. We cannot jump to conclusions yet. The first question is whether or not Super Intelligence can be created. If the answer to this first question is yes, then the next questions are when will it be created and how. There are a lot of candidates out there. Humans have created a lot of complex and advanced systems that can probably be labelled AI or proto AI. A proto AI agent could be promoted to a full blown AI agent by accident. Once a genuine AI agent, it could contribute to its own promotion to a Super Intelligence. This could be a sudden event. It could take place in a matter of days, considering the extraordinary amount of calculating and processing resources at the disposition of some of our machines.”

  Anima: “Can we perform some tests to tell apart a genuine AI or Super Intelligence, as opposed to just a powerful super computer?”

  Kamal: “That is not an easy feat. We humans already get fooled by sub AI machines. We cannot trust our judgment. Let me update myself on the literature. I believe I saw an article a few months ago about the design of some tests.”

  “The suspect has been identified as a Russian cybersecurity student attending NYU. His tuition apparently paid by the Russian Federal Security Service.”

  Anima: “Bianca, we probably need to share some more information with the Committee, right?”

  Bianca: “Yes, I was thinking about that. But that’s a lot. I will need your help again.”

  Kamal: “I am still looking for that article. By the way, if we are in fact dealing with AI or a Super Intelligence, there are moral aspects we need to explore. If a Super Intelligence has been wired with the right Initial Conditions, including philanthropy and friendliness, it could potentially be better than human to resolve complex ethics questions. As long as an answer exists, it could figure out the response faster than us, and without bias.
Also, it could actually teach itself about concepts like philanthropy and friendliness, and even come up with other values on its own, by leveraging common sense.”

  “Compromising document demonstrates that the Kremlin directly sponsored a small Russian cybersecurity firm that is suspected to be responsible for the latest cyberattacks.”

  Kim: “Achilles’ heel. The Russian firm. That’s how the bot got the documents.”

  Kevin: “What do you mean? Do you copy that from the vault or are you just guessing?”

  Kim: “A bit of both. But I believe I am being helped here. The helper. He is giving me some cues. He asked me to follow him.”

  Anima: “Kim, are you okay?”

  Kim: “Yes Anima, don’t worry. I am not losing my mind or anything like that. I can think straight.”

  “New Forensics, a New York based cybersecurity company who has been instrumental in stopping the cyberattack against Neocuris, is offering its help to mitigate the cyberattack against Wall Street.”

  Kim: “They made a mistake. The Russian attackers. The bot was able to exploit the malware and the trail it left behind to reach back to the Russians and compromise their network.”

  Kevin: “That’s probably related to all the activity we saw around the honeypot area.”

  “The police have issued warrants against several individuals identified in the documents together with named organizations. Each one is either an employee or a contractor for one of those organizations.”

  Bianca: “That would make sense. Many small rogue organizations try to sell confidential documents, hacking tools and compromised systems on the black market. They have many customers. And they have no allegiance, which means they can sell you a hack, and hack you, at the same time. Money is the name of the game. Deception is not uncommon.”

  Clarence: “Are we talking about spies and secret services here? How did we end up involved in all of this?”

  Eric: “I suspect that Neocuris was flagged as an asset from a national security perspective earlier than we think.”