Conception: Book One of Human Dilemma

By Scott Sibary

2038: After recovering from an AI-engineered pandemic and the economic collapse that followed, nations agree to create a World Council to be assisted by an early form of artificial general intelligence.
Dr. Solveig Kleiveland, an introspective, fatherless, and patriotic 34-year-old computer scientist, is sent with her Norwegian team to Beijing, China, where the AGI is being assembled. She finds that achieving consensus with her male colleagues, some older and more experienced, is her first challenge. Each of them loves their country, but with different views of how it should fit into the world scene.
Her team’s mission is to integrate into the new AGI the vital codes, algorithms that incorporate ideals from the Asilomar Principles and seek AI alignment with human values. Included is also her own invention: a Protection Lock that should guarantee the sanctity of the vital codes.
The senior Chinese project manager, Dr. Deng AnDe, is a reflective and unconventional thinker. Loyal to many of the ideals in his country’s spiritual traditions, he is utopian-bordering-on-naïve. He hopes China will not be just another destructive force in world politics, but make good use of its brief opportunity to take center stage in world affairs.
Solveig finds herself opening up to AnDe, but then is torn by doubt. She tries to see past cultural and gender biases to determine if there is enough common belief in what they are working towards. Yet she learns she is not the only one, even on her own team, to be skeptical about the project. At least one person is working to undermine the project’s success. She must decide: ferret out the dissenters, or side with them. And she must make sure the key component she contributes will not accelerate the AI arms race, nor help create the most powerful weapon ever.
Likewise, AnDe must resolve his own dilemmas. He finds that his view of Solveig has secretly morphed into an agitating but inspiring presence in his mind. He draws from his training and from the diverse wisdoms of his four grandparents to devise a way through the web of conflicting demands from within and without.

• Language: English
• Publisher: Scott Sibary
• Release date: Feb 10, 2019
• ISBN: 9780463950500

Scott Sibary

Scott Sibary is a native of California, and a citizen of both the US and Canada. He attended the University of California at Berkeley, and also spent one year at the University of Bergen, in Norway. After earning a bachelor’s degree in Scandinavian Languages and Literature, and then MBA and JD, he practiced law in the San Francisco Bay Area, and taught as an adjunct professor at UC Berkeley. He spent most of his career teaching business law at CSU, Chico. His extensive travel experiences, especially in countries economically less fortunate, have contributed to his humanist and internationalist perspectives.In addition to writing, he pursues outdoor activities like hiking, tennis, bird watching, and even the eternal ‘do list.’ "I find myself constantly balancing desire for the disciplined energy to “do”, with the need for equanimity: the sense that being is, in itself, wondrous."

Book preview

Conception

Book One of Human Dilemma

by Scott Sibary

Copyright 2019 Scott Sibary

Smashwords edition

Cover design by Christopher Moisan

Thank you for downloading this ebook. This book remains the copyrighted property of the author, and may not be redistributed to others for commercial or non-commercial purposes. If you enjoyed this book, please encourage your friends to download their own copy from their favorite authorized retailer. Thank you for your support.

A minimum of 30% of the cost of this book (about 50% of the author royalties, varying with different retailers) will be donated to non-profit organizations that work to promote AI alignment with human values (such as those listed in back of this book. The first donee organization will be The Center for Human-Compatible Artificial Intelligence at UC Berkeley).

Disclaimer: Many of the names (forenames or surnames, never both) of the active characters in this book are coincidentally names of people known personally by the author. The use of those names in this book is solely for literary purposes. The active characters in this novel are fictitious, and any resemblance to real persons is unintentional and coincidental (unntatt, kan henda, Rolv). References to real or historical persons are intended to be accurate, and any inaccuracies are unintentional.

Table of Contents

Prologue

Chapter 1

Chapter 2

Chapter 3

Chapter 4

Chapter 5

Chapter 6

Chapter 7

Chapter 8

Chapter 9

Chapter 10

Chapter 11

Chapter 12

Chapter 13

Chapter 14

Chapter 15

Chapter 16

Chapter 17

Chapter 18

Chapter 19

Chapter 20

Acknowledgements

About the author

Links to relevant organizations

Prologue

Decisions, and decisions.

The day began with the clarity of a sky blown clear by north winds, autumn sunlight illuminating the west wall of her office, tea and coffee mugs steaming on the desk, and two minds fresh, buzzing and intent. Then, as the sunlight rose higher, the spotlight disappeared, the mugs stood empty, and the pace quieted. Yet the two persisted, driven by the career-changing proposal from the day before.

A computer screen on the desk displayed part of the code for a new set of algorithms she had dubbed Protection Lock. Its purpose was to guarantee the inalterability of the vital codes—codes aiming to ensure that an artificial general intelligence would remain altruistic, or at least benign. With completion of what the Chinese were calling the world’s first AGI scheduled for as soon as January 2039, Solveig had only a few months to prepare the Protection Lock for trial integration into that system, and only one month before leaving for China.

If she said yes.

Solveig Kleiveland leaned back, her reddish-blonde hair falling free behind her shoulders. Her well-worn cardigan with pewter buttons hung open. She’d been able to make just enough room for her slim figure to fit beside Lars at her desk without bumping their knees. Her younger colleague, in jeans and tight-fitting knit T-shirt, was similarly lean and as fit as a practitioner of modern dance.

It has to be kept entirely confidential, she said.

None of it open code?

The Chinese are keeping the code for their immune system confidential. You, me, and the other team members from Norway will be the only ones with any access to the Protection Lock.

And you, Solveig, will be the only one with access to the part of the code that updates the Lock, assuming you’ve effectively secured the Lock itself?

Right. It’s enough that the vital codes will be open source, as they’ll have to be if the new World Council is to trust the AI. And that’s assuming enough nations continue to back the Indian Proposal, and the Council becomes a reality.

That’s the impression I got from the interview. Only, you haven’t yet agreed to go on the mission, right?

Right.

They sat in silence. Solveig took long, deep breaths, eyes focused on infinity, like a distance runner who has completed one training lap and pauses to consider a second go.

Two soft raps sounded on the door.

Come in! she said.

Rune Andersen, the Director of the Institute for Humanistic Technology at the University of Bergen where Solveig and Lars worked, took a step inside the office. Good morning!

Good morning, both replied.

He closed the door behind him. His necktie and open sports jacket swayed left and right as he swaggered around to the opposite side of the desk. He raised his palms as if to beckon, mouth hinting at a smile.

You’ve come for my answer? Solveig asked.

I need to ask you an additional question.

Wait, Rune. Don’t tell me . . .

He raised his eyebrows.

She shook her head. "Don’t tell me you want me to be leader of the mission?"

He nodded.

Why didn’t you mention it yesterday? Did someone else turn you down? Like Rolv Drammen?

No. The committee didn’t discuss the question until we were done with yesterday’s interviews.

Why not Reidar Hagerup? He and Rolv are older, and have loads of leadership experience.

Thirty-four isn't old enough? You’ve led research teams and taught classes, and it’s only a team of six.

Huh! She coughed. "I’ve led one graduate seminar, and taught an introductory course one semester—not very successfully, in my opinion. And it was you who selected me to lead your small research teams. That’s hardly preparation for being the point person for a semi-secret international mission with potentially gargantuan impacts!"

I’ll grant you that, Andersen said. Yet everyone on the committee, including the representative from the Foreign Ministry, thought you were the best choice. The design aspects of this project are more important than the relationships with the other scientists.

Or even with their leader?

Dr. Deng AnDe? Don’t worry about him. He has an excellent reputation for getting along with people, and he’s assembled a whole array of brilliant minds to work on their project. But he himself doesn’t have your technical genius. He raised his hands questioningly. Yesterday, Solveig, you seemed enthusiastic about the mission, open to anything. What’s your hesitation?

Her focus moved from Andersen to the three photos on the wall. The first portrayed six middle-aged women wearing the traditional ceremonial dress of her home island of Osterøy. The feet of two of the women seemed to rest on a thick black line drawn on the photo with a marking pen. The second was a poster-sized aerial of Osterøy with the name of the hamlet of Hauge bright from a yellow highlighter, and farthest from the door hung a framed portrait of the Crown Princess of Norway.

Things happen at night, she said. One gets shaken up.

Oh yeah. Andersen shook his head and took a deep breath. Don’t we all know. He followed Solveig’s gaze to the first photo on the wall, rubbed his chin and then moved to look out the nearby window with a view over the Bergen harbor.

Actually, she said, and he swung back around.

Solveig glanced back and forth between the two men. I don’t think I’d want to go if I weren’t the leader. The Protection Lock has to be handled correctly. And there are so many contingencies! What if the systems won’t merge?

That’s their problem. We just do our best.

What if they merge, Solveig said, but don’t honor the vital codes? Or their system penetrates the Protection Lock? Worse, what if we hand over our technology, but the World Council is never convened? We might be leaving the Chinese with a dangerous tool, maybe a weapon.

Lars leaned back, one leg crossed over the other and angled towards her. He looked at her with a twisted smile. Ah, so that’s it.

I have to make my own assessment of the risks, she said. I can’t make a knee-jerk reaction to go ahead just because the government deems it best. I bear responsibility, and so do you.

He kept smiling as his hanging foot swung forward to tap her in the back of her calf.

Lars! She gave a tired laugh and shook her head. I’m serious.

Of course. But your Protection Lock is useless by itself. What purpose can it have other than to play a role in a larger system?

"Which system? Whose system? That’s my worry."

Lars threw up his hands. Either the people on our troubled, shrinking planet cooperate more closely, or else . . . He brought his fingertips together. Tell me, can you prosper in a personal relationship if you don’t cooperate? The closer the relationship, the closer the cooperation. It has to be, right?

That’s different. Individuals always have choices—so many alternatives. You should find a good match, eventually.

"Easy for you to say, you who ends things when you find something’s lacking. My situation is more like the international scene, with its limited number of nations. I don’t have many men to choose from, so I’ve embraced the spirit of compromise. He slapped the desktop with his hand. And that’s why, as your closest colleague, I’ve told Dr. Andersen I’m willing to join the mission. And you?"

She leaned forward to rest her chin in her palm. In the quiet of the room, the buttons on her sweater swung against the edge of the desk with the resonance of a ticking clock, winding down.

I share your concerns, Solveig, the older academic said. I think we all do. He put both hands on the far edge of the desk, his shoulders jutting forward and his eyes beseeching. Yet we need to know your answer. Will you be involved? Will you lead? It’s time to decide.

The mind-deadening grey paint on the smooth concrete failed to hide the past. Those working in the old building of the Beijing health department knew what the main room of the basement had been used for. Empty since then, it would soon be given life. Thin, hollow walls had newly divided this subterranean world into several smaller rooms: conference rooms, a computer room, a guard room, a toilet room, and one larger, leftover area as a general meeting room that offered the only door in and out.

Dr. Deng AnDe stood alone, a tall but otherwise unremarkable statue of an everyday, forty-year-old man, and waited. The café where he’d just eaten a quick lunch had been playing the kind of dance music he liked: a fusion of swing and recent Chinese pop that had added to his energy and kept him moving fast, until he reached the basement. He glanced at his phone. He’d arrived twelve minutes early. He figured his boss, director of the Division for Artificial Intelligence within the Ministry of Technology, would probably arrive about one minute early.

The arrival of his boss should feel like a formal endowment of authority; AnDe could then feel in charge of this new job site. The transfer of jurisdiction over the basement facilities to the Ministry of Technology had been a mere technicality, but it symbolized a proactive step forward. At a time when many foreign governments remained as unstable as slowly spinning tops, his country would accept the leading role. It would, AnDe hoped, guide the world away from Armageddon and devastation, and safely into the future.

He could feel the weight on his shoulders.

He straightened his tie, carefully brushed down the front of his suit jacket with his palms, and took slow and deep breaths to let the tension flow out of his body. His feet began to wiggle in his shoes, and he felt his throat wanting to hum a tune. The quarter turn he took to look down the inner hallway resembled a dance step.

Facing the computer room where technicians were working, he again became a statue. In that room was assembled the hardware for the operating system of a new generation of artificial intelligence: an intelligence his teams would be creating with the participation of a small group from Norway.

He shook his head. How had that country managed to persuade China to cooperate on the Indian Proposal? India and the US had not. Norway’s influence seemed out of balance. Was it because the United Nations and the World Trade Organization had proven ineffective at facilitating significant international cooperation, beyond the treaty organizations they ran?

Perhaps his boss could throw more light on the question. It seemed to AnDe that the Chinese government had come to see international cooperation as necessary to prevent engineered disasters, like the influenza virus in 2029 that had killed nearly a billion people.

AnDe completed a scan of the site that would absorb the main focus of his energy for the next several months and was returning to the large meeting room when a familiar figure entered from the main building hallway. Director Liu, a man in his sixties, wore a casual business suit but walked as though he were in charge.

Good morning, Division Director Liu, AnDe said as his boss approached.

Senior Project Manager Deng AnDe, thank you for offering me a tour of your new facilities.

With pleasure, sir.

AnDe led him around the large room, and then paused to glance into the guard room. There, a uniformed watchman sat at a table below a bank of screens. The two continued past the three empty conference rooms and to the door of the computer room. That room bustled with activity. Several white-coated technicians were stringing wires, moving equipment, and testing circuits and routers. One person ran a vacuum cleaner around the corners of the room. AnDe and Liu looked through the door only long enough to receive a few over-the-shoulder glances, then turned away.

You mentioned there was something you wanted my insight on, the elder said.

Yes, sir. There’s something I am curious—or you might say, confused—about. Our government had rejected the Indian Proposal and offered some criticism of the proposed structure of the World Council. But I don’t think that was the main concern. I put more weight on the distrust our government expressed about using an AGI as an official advisor. It seems to me that artificial intelligence has acquired a bad image: people in our country blame the pandemic on the AI that engineered the virus.

Yes, rather than blaming the dictator who thought it would help squash the drought-triggered rebellions in his country.

Or the arms producer who sold him the drones to deliver that plague, even if it was not supposed to be so deadly. Despite all that, our government now seems to be endorsing the use of AI.

About that I have only theories. When a nation is clearly the most powerful, it tends to resist any form of internationalism that it doesn’t control. We saw that with the US government on the subjects of arms control, climate change, and human rights. The plague, as some call it . . . it hit our country hard, but what were bumps and stumbles for us became a free fall for them.

And so their government fell to a coup.

Sort of. I suspect power was already being lost to private military contractors employed to quell the civil unrest. The coups by the traditional militaries in the US and UK restored a kind of government order, and they later handed power back to an elected government. But by then their economies had crumbled and China was the new economic leader.

That’s what I’m getting at. Once China became the most powerful, why did our government change its position and endorse a new international organization?

I wouldn’t call it a change. After we saw how much harm AI could cause—or I should say, could be involved in—it made sense not to pursue an internationalism that relied on AI. For the time being, that is. But much progress has been made on the design of vital codes. There are now several versions that might resolve the conundrum.

The conundrum, AnDe reflected, was the great puzzle that if left unanswered too long could lead to the end of humanity. The more free-thinking the AGI, the more difficult it would be to direct. If too controlled, it would be too slow and limited to achieve significant new results; but if too dutifully focused on a particular result while remaining free in its choices, it would likely cause collateral harm. Yet regardless of the conundrum, the drive to create powerful AGI would not be stopped.

"Why endorse this version?"

Ah, I’ve known you too long, AnDe, not to see there is something more you’re driving at. The Protection Lock?

AnDe sighed. Yes. We’re told it keeps the vital codes sacrosanct. We’re told it’s very strong, and in the future it could be readily transitioned for use in a quantum computer. But is there something else that makes this Protection Lock so attractive?

I don’t know, unless it’s simply that it works, in which case—Liu held up his index finger—"it will protect the vital codes and guarantee that the conundrum remains answered. He dropped his hand. Or it may be just that those at the top have decided the proposal is a venture to investigate, to see what will work. They might like that the Norwegian modification to the proposal has the World Council sit in Beijing rather than Delhi." He offered AnDe the smile of an insider.

A few paces from them stood a woman in custodian’s overalls, her back and one ear towards them, polishing two large, brass letters on a door: WC.

Liu looked from the door to AnDe. Only one?

AnDe sighed again. Might as well drop the subject and accept the answers one gets. It’s unisex. Outside, by the elevators in the main hallway, are two more: a men’s and a women’s.

The letters gleamed, and AnDe began to chuckle.

Liu stared at him.

I apologize, AnDe said. I thought it humorous that the abbreviation could be the same for the World Council.

The hand of the woman paused, and the squeaking of the cloth, still pressed against the letters, ceased.

Surveillance? he asked his boss.

Liu waived a dismissive hand in the direction of the janitor. An old dilemma, control versus freedom. And here we engineers are trying to build an optimal middle path.

Don’t security officials understand that too much control leads to a decrease in productivity? AnDe asked.

Standardizes it, control does. Gets rid of rotten eggs but dims the stars. There are five stars in the Chinese flag, and we need all of them to shine together.

Yes. And with challenges like our worsening climate, the whole world needs all of its stars to shine, General.

General, yes. Liu paused, as if reflecting. Did you know I only reached major in the army engineers? They bumped me up in the Reserves. Because of loyalty, no doubt. Yet if they ever took me away from my position as director of the Division for Artificial Intelligence because they needed me full-time in the Reserves, it would already be too late. None of this would matter. The slight gesture of his hands suggested he meant a world beyond the room in which they stood.

That’s why I believe this mission is so important, AnDe said.

Youthful devotion, Liu said. That’s good. You will need it.

Chapter One

What am I doing here?

She stood in the middle of a hospital ward, its rows of beds extending into the distance like a battlefield cemetery. Moaning rose around her like a dirge. She looked at the faces on the beds. None was distinctly recognizable, and they changed even as she stared at them: familiarity slipping away before it could be grasped.

A loaded gurney, sheet draped over the corpse, came straight at her. She took a step away, and it stopped beside her.

A hand shot out from under the sheet, grabbing her wrist. Then, like so many before, the scene faded away as her eyes struggled to open.

Solveig turned to the clock on the nightstand: half an hour before they should gather in the hotel lobby. She flew into the shower, the sweat on her forehead cooling her as she moved. Another nightmare to rinse away.

Half an hour later, she had finished an order of oatmeal from room service and stood in the middle of her room, holding an empty celadon tumbler in her hand. Her phone on the bed flashed and rang. It recited a message that the liaison had arrived and that her team was assembled and ready. She sent no reply. Instead, she raised her empty teacup to her face, as if it still held something of import: something that might decode a maze. Her fingers cycled the vessel over her palm while she examined a style admired for centuries.

This hard, transparent glaze exposes a world of underlying cracks, she thought. I wonder if I could decipher a pattern.

She rotated the cup in the opposite direction, as if a hidden pattern might reveal a clue to an intricate and mysterious insight: a metaphor for history.

She returned the cup to her breakfast tray and stepped over to the blackout curtains. Parting the heavy layers with her left hand, she faced the inclement weather, then stiffened from a renewed feeling of rebellion.

It’s all you have, the protesting voice insisted. What if the merger doesn’t work? You’ll be nothing, Solveig Kleiveland! You’ll be meaningless: a career wasted. And if it works the wrong way . . . Oh my god. It would be better to abort, or else you’ll be far, far worse than nothing.

Yet, the temptation! A chance to go beyond what they’re trying to do: how can you not?

She shifted her weight back and forth between anxious feet as a different concern ambushed her from its perch in the back of her mind. She’d let the turbulent currents of world affairs free her from a relationship that had gone stale. Like parting company with a temporary travel companion while disembarking at a busy airport, she’d simply let go of something that no longer had meaning. Yet in China, she’d just be getting older.

Forget that, Solveig. You’re not here for relationships.

She remained at the window, hoping to determine which of her objections was the greater culprit, or the greater sage, behind her doubts. As she sought clues, the weather merely reflected the question back at her: grey drizzle obscuring the view, sunlight diffused by an overcast. Such a scene could inspire any number of conclusions, driven by one’s inclinations. She stood rigid like an unwilling bride about to be wed to a groom she had never met. Outside was the groom: the giant metropolis of Beijing, with its size hidden by the mist and its power only hinted at by the glistening expanse of tall buildings. She had to wonder whether such a world genuinely wanted anything a poor country-girl from a small nation had to offer. And if it did, would that make it any better?

She knew this kind of cooperative project was the nature of her work. Her part would have no utility until integrated into the work of others—of strangers she would have to work with intimately. Strangers she was about to meet.

She closed her eyes.

Sometimes it works out and sometimes it doesn’t,