This paper presents some components of the learning system within the Independent Core Observer Model (ICOM) cognitive architecture as applied to the observer side of the architecture. ICOM is uniquely designed to continuously enhance its problem-solving capabilities through a mechanism that integrates feedback from past experiences to optimize future actions, generate new actions, and extend the system functionality on the fly to perform new functions as the System determines is needed. Central to this System are problem identification, proposed solution generation, and implementing solutions through testable models of an action that directly inform new task achievements or goal settings related to those tasks. By embedding this functionality within the ICOM architecture, we enable the System to adapt and extend its functionality dynamically over time. Integrating continuous learning and adaptation with the rest of the System provides a powerful tool for self-evolving artificial intelligence systems. This approach improves the System’s efficiency and effectiveness in handling diverse and unforeseen challenges.
The problem-solving model generation system specific to actions that the Independent Core Observer Model (ICOM) System wants to take within the ICOM cognitive architecture [Kelley] is the core purpose of the paper, providing the basics for how these systems can be used. This System is designed to continuously learn and adapt to new tasks, improving its problem-solving capabilities and allowing it to grow and extend its functionality dynamically. The critical components of this System include a feedback mechanism, problem-proposed solution generation, and implementation of new tasks or goals in learning actions. By combining these components within the ICOM cognitive architecture, we aim to create a powerful and flexible problem-solving system that paves the way for more advanced and adaptive cognitive systems in ICOM.
NOTE: figures and conclusion redacted. see the research gate copy for additional details. http://dx.doi.org/10.13140/RG.2.2.17125.82406
Independent Core Observer Model (ICOM) Cognitive Architecture
The Independent Core Observer Model (ICOM) cognitive architecture is a cognitive framework that aims to create a self-aware, conscious artificial intelligence system that is both sapient and sentient and makes all its decisions based on internal subjective experience. ICOM does this by implementing several vital patterns or theories related to cognition, including Integrated Information Theory [Balduzzi], Global Workspace Theory [Baars], Computation Theory of Mind [Rescorla], Hierarchical information theory [Ahmad], Conceptual Dependency Theory [Schank]; where Global workspace theory, as articulated by Antonio Damasio [Damasio], is the primary foundation the ICOM architecture is based on. Please see existing research for more information about ICOM in general [Kelley]; while borrowing from all of this, ICOM uses the ‘Abstract Theory of Consciousness [Kelley].
ICOM is designed to be a modular and flexible system that can easily be modified or have components switched out. It consists of an observer module responsible for processing sensory information, executing decisions, building knowledge graphs, generating models of actions, and otherwise interacting with the external world. The core module calculates a model of internal or simulated emotional states and how they relate to any new input. When the core simulates positive values related to a given knowledge graph, the observer ‘sees’ that and tries to execute on that action if a preferential action model is present on any given graph sent to the ‘core.’ [Kelley, Twyman].
In ICOM, self-reinforcement [Sutton] happens as results from actions are stored as graph models in the overall context graph database, and the edges of the graph are created only on subjective state models by intensity. Actions that have positive associations are more likely to be selected and more likely to further enhance the positive nature of the results associated with a given action. This process works for positive and negative reinforcement based on action results that bias the System toward any choice, goal, or motivation.
Problem-Solving Model Generation
Problem-solving model generation is a process that involves continuously adapting and evolving solutions to complex problems. There are a lot of approaches, from adaptive heuristics, meta-learning, self-play/simulation, transfer learning, evolutionary algorithms, and more [Russell]. This approach recognizes that problems are not static and require ongoing attention and innovation to address effectively.
ICOM attempts to address this in the context engine part of the System, where an initial input interpretation is converted into a knowledge graph model. The knowledge graph is built based on relationship vectors and directly related models from preexisting graphs. It will try to identify possible actions and look to see if an associated action exists in the form of a model. Creating or building a new action model goes through the following process in ICOM, which occurs primarily in the observer engine after the core simulated subjective analysis.
Figure A – Problem Solving Model Generation Diagram.
In this diagram, we can see the recursive process of building out possible steps. Based on the results, the action model will be modified the next time the System requests this action if the results are not optimal.
This can be done without LLMs (Large Language Models) [Meyer]. In initial versions of ICOM, this process was limited and narrow in scope, where LLMs get us to something approach the broad generic functionality as seen in humans. LLMs get us farther along with natural language processing, and idea generation is more refined by orders of magnitude when combined with the knowledge graph from the context engine. The usage of LLMs is incidental to ICOM. The LLM gets our final models to be more refined going into processing. It allows a level of refinement for new tasks that were not possible previously. LLMs are a technical catalyst for complex problem-solving systems that are done from scratch using this method. That said, it should be noted this is a step away from how the human mind does it to a large degree. Even though general functionality might be the same or similar at a high level, or the subjective experience of the System might be the same at a high level, the implementation details are very different. This difference is relevant to what we know how to do currently and does not seem to directly relate to how humans do these sorts of tasks in detail.
Implementation of New Task Actions
In current implementations of ICOM, there is a built-in predilection for action, meaning that if there is a chance of a positive outcome, more likely than not, the System will try to take a given action because it automatically associates positive values to actions over not taking action. Still, most often, such an action might not be defined, which would be done by the Observer after the System has selected a given action to take. Suppose a given step required to take a given action is unknown. In that case, the System will try to build that step dynamically as either an ECMA block (JavaScript or Typescript, etc.) or other transformation-based logic (XSLT or other XML variant) defined in the overall action object model (“defined” in JSON or XML) or if there is an existing XML definition for something that seems to be similar. The details of a given action generally will not filter up to the context database or the conscious awareness of the System. However, they will still be stored in the context graph. The System will draw context graph data and use a deep neural network (DNN) or Large Language Model (LLM) to generate what that step might be. This is done at the observer engine level or by a part of ICOM called “the Observer Engine.” Then, the Observer Engine takes the action, and the results are passed back to the context engine (also part of the Observer) to reinforce the action based on the results. When actions are taken a second time, those past results will largely determine what action is taken the second time or if modifications need to be done.
Continuous Learning and Adaptation
Learning based on action results in ICOM is done by analyzing the results of generated actions and subjectively reinforcing those actions taken or not taken in the context graph based on results based on how well the results match subjective contextual data initially assigned to the model. Each time an action is taken, the System will try to find errors and correct or improve steps for a given action, especially if the results are not “good” as defined by the System.
Building on the adaptive and dynamic nature of the ICOM system, this approach significantly enhances the development of robust and flexible problem-solving capabilities. ICOM has evolved into an increasingly adept cognitive system by integrating real-time feedback mechanisms and continuously refining its action strategies. The use of deep neural network models, including Large Language Models, allows for the generation of sophisticated, context-aware steps that are not predefined but are dynamically constructed based on immediate needs and past outcomes. This capability makes ICOM flexible and adaptive to new, unpredictable scenarios, pushing the boundaries of traditional problem-solving systems. Moreover, the System’s ability to learn from previous actions and adjust its behavior based on emotional valences attached to outcomes positions it as a forward-thinking model in adaptive cognitive systems. This ongoing cycle of action, evaluation, and reinforcement leads to refined decision-making processes that can handle complex, evolving challenges, setting a new standard for intelligent systems operating independently while aligning with desired emotional and contextual outcomes.
Consider the following example.
The example system incorporates three main components: an XML Object Model for structuring task representations, ChatGPT Interaction for generating detailed action step generation and corresponding code, and a Neo4j Graph Database for storing these models to facilitate learning from previous executions.
The methodology unfolds through several phases: initializing the System with an XML document, decomposing tasks into detailed steps via ChatGPT, determining the appropriate execution strategy (JavaScript or C#) for each step, and executing and storing these actions in a graph database.
Note that this is a simplified version of how the ICOM research implementation does this.
In this example implementation, we highlight the separation of concerns and clarify the functionality of each code block. However, we will not cover library details core to the language C# used.
class Program
{
static HttpClient client = new HttpClient();
static async Task Main(string[] args) { }
Figure B: Object Setup
This segment defines the Program class and its main entry point. The HttpClient is initialized here to facilitate API interactions throughout the application. The Main method is marked async to enable asynchronous programming, allowing the program to run tasks concurrently without blocking the main execution thread.
String xmlActionModel = "<ActionModel><Task>Write a blog post</Task></ActionModel>"; XmlDocument doc = new XmlDocument(); doc.LoadXml(xmlActionModel);
Figure C: XML Initialization
An XML document is initialized with a predefined structure containing a basic task. This XML model serves as the foundation for generating further action steps. The XmlDocument class is utilized here to load and parse the XML string, setting up the initial structure of the action model.
XmlNode taskNode = doc.DocumentElement.SelectSingleNode("//Task");
String taskDescription = taskNode.InnerText;
Figure D: Task Node Selection
The task description is extracted from the XML document. This step involves selecting the “Task” node from the document and retrieving its text content, which specifies the task that needs to be broken down into executable steps.
XmlNode stepsNode = await ProcessSteps(task description); doc.DocumentElement.AppendChild(doc.ImportNode(stepsNode, true));
Figure E: Recursive Step Processing Initiation
This code initiates the recursive decomposition of the task into detailed steps by calling the ProcessSteps method. (See associated .cs file referenced (Kelley).) The resulting nodes are then appended to the main XML document, enriching with executable steps and sub-steps.
Doc.Save("ActionModel.xml");
await SaveToGraphDatabase(doc);
ExecuteActionModel(doc);
Figure F: Saving and Executing the Action Model
Finally, the enriched XML document is saved locally and stored in the Neo4j graph database. This persistence allows for tracking the evolution of action models and learning from past executions. The action model is then executed, potentially involving running JavaScript code or C# functions as defined in the earlier steps.
These code blocks provided a relatively simple but comprehensive C# program that integrates an Artificial General Intelligence (AGI) system with an AI language model, specifically ChatGPT, to dynamically generate and execute action steps based on given tasks. It begins by loading an initial XML task description. It then recursively queries ChatGPT to break the high-level task into more detailed steps. For each step, the program determines whether it should be executed using JavaScript or a C# library based on the nature of the task.
In the ICOM project, JavaScript is typically chosen for web-related or interactive tasks generated by the System. At the same time, C# is selected for the actual code and engineering base components. This decision-making process is encapsulated in a method that assesses keywords in the task description. After generating the appropriate code snippets, the program embeds these within the original XML model, enriching it with executable steps. Finally, the enriched XML model is saved to a Neo4j graph database for persistence and context-aware learning, and the steps are executed accordingly. The System thus enhances its functionality autonomously by learning from each task execution, enabling it to handle increasingly complex scenarios over time. Using JavaScript (ECMA Script), we can add code without recompiling the System, and ICOM is written in C#.
Leveraging Generated Action Graph Models in ICOM: A Recursive Approach
The example above explores the novel approach used in ICOM, where an Artificial General Intelligence (AGI) system dynamically enhances its functionality by generating and refining action models using a recursive methodology that can add additional functionality without recompiling. By interfacing with an external AI language model (e.g., ChatGPT), the System autonomously develops executable steps in response to tasks defined in an XML-based (Extensible Markup Language) object model (which can be defined elsewhere in an overall cognitive architecture). In ICOM, these steps are stored in a context graph database, facilitating iterative learning and refinement over time. This methodology allows ICOM to self-improve and adapt to complex scenarios by continually updating its action models based on new information and outcomes.
The increasing complexity of tasks that AGI systems are expected to perform necessitates a design that can evolve and adapt dynamically. Traditional static programming methods are limited in flexibility and scalability when faced with novel or changing tasks. In contrast, a system that can generate its operational steps and continually refine them offers significant advantages in terms of adaptability and efficiency.
Recursive Refinement: Each step is further broken down recursively, with the System querying ChatGPT for finer sub-steps, ensuring comprehensive coverage and detail in the action plan.
Execution and Learning: The System executes the generated JavaScript or C# code, records the outcomes, and updates the graph database with the new data, thus refining its models.
Let’s look at cases using ICOM for problem-solving. This case was created by a research implementation of ICOM using code similar to the previous section. Let’s look at a logical breakdown of processing to generate an action model from scratch, starting with the following diagram:
Figure H: Visual Model of Graph
This is a visual model of a knowledge graph created by the context engine in ICOM after the System decided this would be a good thing to take action on. The System realizes it has no existing models and no meta-information about this task. Given that the System has decided this is a good thing, it will process both the information it needs and the steps to execute this action. Let us look at that information added to the graph of the ‘action’ model the System builds.
Figure I – High-level Steps and Information
For the computer to execute these, it must first figure out the additional data. In ICOM, the System will first identify what it can infer and what tasks it needs to do to get or infer the rest of the required information.
Figure J – Step 3 Identify Steps to Get Information
In this case, “ASK” is a defined task. The lack of information blocks overall action execution, so given the latest data, the System will execute the ‘ASK’ tasks to get the additional information needed. Now, if this information had been provided or otherwise inferred by past execution, the steps could be different. At this point, the graph would be used as a model for running the ‘ask’ action model 5 times to gather data.
In executing this experiment, the System identified additional requirements and equipment to acquire. It created steps for ordering that equipment and hiring people to set it up, including system design requirements. One preexisting control the System has is using Node.js as a step and a library called “Puppeteer.” The system-generated code can be part of the overall acquisition process for a single product on an e-commerce site like Amazon. In the experiment instance, the generated script using Puppeteer and node.js could order a single product from Amazon and validate if it was successful, and this was added to the details of a given step in the overall action graph model. This could lead to a discussion of predictive coding, as explained in Karl Friston’s [Friston] research.
In this example, it took 378 steps that included individual calls to the graph/vector database and calls to the LLM in question to build a massive tree or knowledge graph that the System could successfully execute and get set on its own to provide a cup of coffee with little upfront information. Given the methodology, this could take some time for a task like ‘making’ a cup of coffee. Still, it demonstrates the ability to do random tasks, develop a model, and problem-solve to a solution.
…
References
Ahmad, S.; Hawkins, J.; “Properties of Sparse Distributed Representations and their Application to Hierarchical 397 Temporal Memory”; 24 MAR 2019; Cornell University Library
Baars, B.; Katherine, M; “Global Workspace”; 28 NOV 2016; UCLA http://cogweb.ucla.edu/CogSci/GWorkspace.html
Baars, B.; McGovern, K.; “Lecture 4. In the bright spot of the theater: the contents of consciousness;” CIIS 2005
Baars, B.; Motley, M.; Camden, C.; “Formulation Hypotheses Revisited: A Replay to Stemberger”; Journal of Psycholinguistic Research; 1983
Baars, B.; Motley, M.; Camden, C.; “Semantic bias effects on the outcomes of verbal slips”; Elsevier Sequoia 1976
Baars, B.; Seth, A.; “Neural Darwinism and Consciousness”; science direct – Elsevier 2004
Balduzzi, D.; Tononi, G.; “Qualia: The Geometry of Integrated Information”; PLOS Computational Biology 5(8): e1000462, 2009. doi:10.1371/journal.pcbi.1000462
Friston, K.; Kiebel, S.; “Predictive coding under the free-energy principle;” Philos Trans R Soc Lond B Biol Sci. 2009 May 12; 364(1521): 1211–1221. doi: 10.1098/rstb.2008.0300
Damasio, A.; “Descartes’ Error: Emotion Reason and the Human Brain”; Penguin Books 2005 ISBN: 014303622X
–––; “The feeling of what happens: body and emotion in the making of consciousness”; Houghton Mifflin Harcourt, 1999.
–––; “Self Comes to Mind: Constructing the Conscious Brain”; New York: Pantheon, 2010
Damasio, A.; Brooks, D.; “This Time with Feeling;” Aspen Institute 2009; https://www.youtube.com/watch?v=IifXMd26gWE
Kelley, D.; “Independent Core Observer Model Research Program Assumption Codex;” BICA 2019, Pre-conference Proceedings: https://www.springer.com/us/book/9783030257187
–––; “The Independent Core Observer Model Theory of Consciousness and the Mathematical model for Subjective Experience;” By ICIST 2018 – International Conference on Information Science and Technology – China – April 20-22nd. (IEEE conference); Year: 2018, Volume: 1, Pages: 396-400; ISBN: 978-1-5386-6956-3; https://www.computer.org/csdl/proceedings-article/icnisc/2018/695600a396/1dUo3atoEAo
–––; “Problem-Solving and Learning Strategies within the Independent Core Observer Model (ICOM) Cognitive Architecture (example CS code);” 061202024; Research Gate; DOI: https://dx.doi.org/10.13140/RG.2.2.20376.40969
Kelley, D.; Twyman, A.; “Independent Core Observer Model (ICOM) Theory of Consciousness as Implemented in the ICOM Cognitive Architecture and the Associated Consciousness Measures;” By AAAI Spring Symposia (AAAI SSS19) – Stanford, CA; March 2019; http://diid.unipa.it/roboticslab/consciousai/; Published Volume – http://ceur-ws.org/Vol-2287/; Published (PDF) – http://ceur-ws.org/Vol-2287/paper33.pdf
Meyer, J.G., Urbanowicz, R.J., Martin, P.C.N. et al. ChatGPT and large language models in academia: opportunities and challenges. BioData Mining 16, 20 (2023). https://doi.org/10.1186/s13040-023-00339-9
Rescorla, M.; The Computational Theory of Mind; Stanford University 16 Oct 2016; http://plato.stanford.edu/entries/computational-mind/
Russell, S.; Norvig P.; “Artificial Intelligence: A Modern Approach;” 3rd edition; Pearson, 2009; ISBN-13 : 978-0136042594
Schank, R. C. (1972). Conceptual dependence: A theory of natural language understanding. Cognitive Psychology, 3(4), 552–631. https://doi.org/10.1016/0010-0285(72)90022-9
Shapiro, L.; “Embodied Cognition (New Problems of Philosophy);” Routledge; 2010; ISBN-13 : 978-0415773423;
Sutton, R.; Barto, A.; “Reinforcement Learning: An Introduction;” MIT Press Cambridge, MA 2018
see the entire preprint draft here: http://dx.doi.org/10.13140/RG.2.2.17125.82406
Leave a Reply