"You've Got to Be Carefully Taught", the Ethical aspect of AI, September 2026
“You’ve Got to Be Carefully Taught”, the Ethical aspect of AI”
September 2026
Disclaimer:
This article is based on my own reflections, concerns, examples, and judgments as a mere user of AI and not as an expert. An AI tool was used as an editorial assistant in structuring my thoughts, addressing possible factual inaccuracies or assumptions and suggesting clearer framing for this post. I take full responsibility for the content.
— o —
Not James Taylor’s “Carolina on my Mind”, but “AI on my Mind”
These days, just like everyone else, I’ve been thinking a lot about AI. Most of the time, it’s about how I do my part to use it responsibly. But what is more concerning for me is its long-term impact and how it could shape society, especially for the younger generation.
In this musing, I wanted to express my concerns that have been creeping since I’ve been watching a lot of AI-related TV shows these days. Yes, I know, I watch too much TV! But that’s the beauty of no longer having a full-time and 9-5 job! Watching “Humans”, “Silo”, and the “Star Trek: The Next Generation” episode, “The Measure of a Man”, where Captain Jean-Luc Picard, defends his android officer and friend, “Data”, highlighted a concern for me.
And it’s that AI or AI systems could become an efficient mirror of the best and worst things we humans tend to repeatedly model.
Life imitating Art? “Humans” and “Silo”, TV shows that make me think
The series “Humans” which ended after 3 seasons, got me thinking a lot. It imagines a world where people could buy human-looking robots or “synths,” powered by advanced artificial intelligence. These “synths” served as caregivers, nannies, assistants, workers, and companions. In the show, before some became “conscious”, they were generally treated as machines that could not refuse, feel, or meaningfully push back. They were quite useful at home and handy to humans. But they were also vulnerable to human control, exploitation, and abuse. When some synths gain consciousness, they begin to act autonomously and form bonds. But some also remembered how they were badly treated. Because of this, human fear kicked in and many humans responded with fear. They tried to control or eliminate them and others struggled to defend their rights.
For me, this show revealed the two sides of human nature. Humans were brilliant enough to create machines that could perform care, comfort, and service in deeply human-like ways. But some humans became fearful enough to resent and destroy the good they had created because it made them feel threatened, replaceable or no longer fully in control.
In “Silo”, however, the dilemma is quite different. In this fictional world, residents are led to believe that a catastrophic event has made the outside world uninhabitable. So, a group of powerful decision-makers created underground silos to preserve selected human communities. To maintain order, these “Founders” established a “Pact” and a system of control that they implemented through surveillance, fear, strict rules, or what they called, “IT/Judicial” authority. They also confiscated “relics” from the past and tightly controlled information to keep residents from rebelling or escaping to the outside world.
“Who Should be in Charge?” - should that be the question?
Together, watching these 2 shows left me with one question. Who should really be in charge? And especially in our AI-age today, if humans are capable of fear, greed, secrecy, and control, even if they’re equally capable of compassion, cooperation, regulation, and moral imagination, how should we govern AI wisely without handing too much power, either to people or to machines?
I know these are fictional stories and not definite indicators of what is to come. And I know that current AI systems are not conscious or sentient. But with AI capabilities seemingly advancing quickly and, especially, recent reports about loss-of-control incidents and increasingly autonomous AI-agent behaviour, these ethical questions now seem less far-fetched to me.
I’m also quite aware that today’s AI systems, autonomous software agents or fictional conscious androids or “synths” are not the same things. But these stories help me think about ethical questions as machines become or will become more capable and present in our daily life. Most of all, they also remind me of how humans respond to power, dependency, usefulness, fear, and control.
So, given how humans can also weaponize AI, and while human governance can also possibly cause distrust, chaos through greed, lies and keeping people in the dark, is it really still the best idea to make humans in charge? In particular, is the phrase, “Human in the Loop” still the right idea? Or should it be maybe changed to “Ethical or Accountable Human(s) in the Loop”, i.e. to also not concentrate the decision-making to one person?
Why this is important?
This topic of responsible AI use matters to me deeply because, as a coach, parent and “rewired” individual who strives to make sound and knowledge-based decisions, I am recognizing how these AI systems are increasingly entering our daily work, healthcare, education, governance, and care. But I’m not sure we, or maybe I, fully understand the impact of what our data, incentives, and behaviours we feed them may be reinforcing while we design them.
After having unpacked this topic with my peer coach, I have realized that my fear is less about AI itself but more about our human nature, particularly the negative one. The show, “Humans”, was quite insightful in that despite the good that synths’ work did for human beings, it also showed how easily humans exploit the good things that they have (ex. nature today), especially those which don’t push back. In the show, humans created something beautiful only to abuse it, until it turned on them. Then because of fear, many supported legal and political measures to restrict or control them. The exact line in the series finale of the “Humans” show that stuck with me was when the protagonist said, "You are a scientist! And you're about to wipe out the greatest scientific discovery in human history! What's logical about that?”.
So perhaps these current reports of AI agents behaving in unexpectedly autonomous and “loss-of-control” ways should be seriously monitored or taken as early warning?
“You’ve Got to Be Carefully Taught”, a song from the Broadway play, “South Pacific”
I am also reminded of the song “You’ve Got to Be Carefully Taught,” from the old Broadway play, “South Pacific”. In essence, it says hatred and fear are learned early and deliberately. That idea stays with me because it suggests that what we repeatedly model and teach (formally or informally), can shape what comes after us. Children watch their parents and think that what they see is the usual and normal way to be. And as they grow up, they consciously or unconsciously, carry what they learn forward.
AI is, of course, not a child. But metaphorically, AI can be seen as a creation or brainchild of humans because it inherits and mirrors the data, incentives, behaviors, and priorities we repeatedly feed into it. We may not even need to deliberately program prejudice or cruelty for AI to reproduce harmful patterns. Biased data, poor incentives, careless feedback, and harmful use of it may be enough to do that. And because AI can scale quickly, I join others in saying that how we shape AI systems today deserve our intentional focus and attention.
Is it really a fantasy?
People are probably saying that I watch too much TV and talk too much about the make-believe. But is it really a fantasy? It is possible that, in the coming decades, more embodied AI systems will be running our factories, caring for our elderly, sweeping the streets, or changing diapers in the middle of the night. There was even one post in LinkedIn some time ago where someone posted something like, “I wish AI can do my laundry and iron my clothes!” In theory, this could really be great, especially for working parents. Delegating heavy, repetitive, and exhausting tasks to machines can give us more quality time with family. And now with humanoids being ready for combat or a race, it might just be a matter of time.
Yet, I fear that our own hardwired survival mechanisms will get the better of us. Human history suggests that we can be prone to fear what seems stronger, smarter, or capable of replacing us, especially when livelihoods and identity are at stake. There have been many examples of human goodness especially during times COVID, natural disasters or wars. But there also sadly many examples today of how humans, as well as countries, not being satisfied with being 2nd best and trying to outdo or dominate each other. In the case of AI, instead of embracing its potential for helping human societies by doing mundane repetitive or dangerous work, people whose work is displaced might respond with anger or resistance and not just from envy but from very real fears about livelihood, dignity, and identity.
I think we also don’t always practice what we preach. We write codes of conduct and ethics. But in our daily transactions, we can be mean, selfish, and have small minds. If AI systems are designed, deployed, governed, or misused by people with bad intentions or unchecked personal greed, the damage could be serious and difficult to reverse.
“Human in the Loop” or “Ethical and Accountable System in the Loop”?
So I go back to what we say these days in keeping the “Human in the Loop”. This might not be enough if the human is fearful, greedy, careless, or unaccountable. Maybe what this statement needs to be is “Ethical and Accountable Humans in the Loop”. Perhaps what we especially need are ethical humans in accountable systems that practice transparent oversight, diverse review, clear limits, as well as shared responsibility.
I believe the shared responsibility and accountability parts are important because it sends the message that Governing Roles have expiration dates, that those in them should not get attached to ‘power”, and that they just can’t do whatever they want.
Other questions also arise like: What prevents “ethical humans” from becoming another concentrated authority? Will a well-designed and accountable AI system be a good check for humans given their dual nature? So should that phrase really change to “Ethical System in the Loop”? And do I have an answer to these other questions? Heck, no! I’m just an AI user who notices things and asks “good faith intentioned” questions so the smarter ones among us can figure it out.
Current initiatives on AI Governance: What would be good to happen?
If we have any control today, it would be to continue supporting initiatives that establish strict policies of AI governance. It’s an encouraging sign to see that the European Union has adopted the AI Act, with obligations taking effect in phases and that the United States is continuing to develop a mix of federal guidance, policy frameworks, and proposed legislation on AI governance.
From discussions and what I’ve been reading, I agree that these governing policies are critical. Clear, non-negotiable boundaries and guardrails are essential. And AI systems should also be designed and tested so they absolutely reject harmful instructions. In general, it would be good to teach AI the standards we value, such as fairness, care, accountability and respect, even when we ourselves fail to treat each other that way.
What might also be good is that AI help us flag risks, slow down harmful decisions, and reflect back our stated values, just like how ICF coaches reflect back what they hear or see. In moments of human weakness when our limited, fear-driven human brains succumb to panic, anger, or unethical choices, a carefully designed and governed AI system might help reflect our stated values back to us, but supplemented with human accountability, transparency and oversight. In the program, “Silo”, however, even with human governance, many still resisted because they were denied truthful information and meaningful participation in decisions that affected their lives.
So what? What if I had my own “Data” like in “Star Trek: The Next Generation”?
Perhaps it’s too much of a romantic and idealistic dream to think that we can teach AI desired ethical behaviour when we struggle ourselves to be ethical. And given some risks that already seem to be present in AI use today like bias, misuse, overreliance, weak governance, and lately autonomous software agents, these developments should make us more serious and intentional about ethical AI use.
As for me, I will strive to be an ethical user to this technology. I commit to only reflect messages or behaviours of compassion, care and concern for others. And if we manage to get to a world with embodied AI or humanoid robots and I end up with my own “Data” like Jean-Luc Picard had in “Star Trek: The Next Generation”, I will strive to model good ethical behaviour and treat it with kindness, curiosity and respect.
If we manage to engrain good human ethical behaviour in AI design, training, deployment, and governance, it can become not just a powerful tool, but a partner that can reduce our burdens while reminding us to stay ethically human. It can potentially help us with tasks that exceed our physical stamina, memory, or attention. And we’ll be able to free ourselves for the simple pleasures of living life with full joy and being human.
But this won’t happen automatically. It will largely depend on what we teach it, what we model, what we allow or reward it. If we want ethical AI, we need more than technical safeguards. We need ethical humans, accountable systems, and the humility to examine how we behave every day.
And if that Broadway song was sung to AI, it should also be, “You’ve Got to Be Carefully Taught”.