How

How does this work?

ThereĀ areĀ fourĀ components to this.Ā  They each communicate slightly differently.Ā Ā 

  1. ServiceNow Instance
    1. Sends REST Request via ecc_queue
  2. Midserver
    1. Executes REST call to the AI InĀ A Box Server
  3. AI In A Box Server
    1. Handles RESTĀ by hitting OpenAI Compliant API endpoint
    2. Handles Streaming Response from OpenAI Compliant API, and passes them along to ServiceNowĀ 
  4. Large Language Model ServerĀ 
    1. Handles REST calls from AI In A Box Server