From 93362b209b7bd1732762f796a9fa42da0671a778 Mon Sep 17 00:00:00 2001 From: "Peter.Morton" Date: Sat, 24 May 2025 12:27:45 -0500 Subject: [PATCH] Adding CQA Retriever --- CQA_Retriever/README.md | 57 +++++++++++++ ..._f47dad8f-5b7b-48c3-97bc-7bba542c99b5.json | 1 + .../GlobalVariable/CQA_Retriever.js | 74 +++++++++++++++++ .../GlobalVariable/CQA_RetrieverSettings.json | 16 ++++ .../_studio_dependencies/HubPackage.json | 5 ++ CQA_Retriever/_studio_dependencies/README.md | 80 +++++++++++++++++++ CQA_Retriever/widget/README.md | 7 ++ 7 files changed, 240 insertions(+) create mode 100644 CQA_Retriever/README.md create mode 100644 CQA_Retriever/_studio_dependencies/ConversationFlowExport/IVA-Solution-Consultants_Peter's-Workspace_Example-RAG-using-CQA_f47dad8f-5b7b-48c3-97bc-7bba542c99b5.json create mode 100644 CQA_Retriever/_studio_dependencies/GlobalVariable/CQA_Retriever.js create mode 100644 CQA_Retriever/_studio_dependencies/GlobalVariable/CQA_RetrieverSettings.json create mode 100644 CQA_Retriever/_studio_dependencies/HubPackage.json create mode 100644 CQA_Retriever/_studio_dependencies/README.md create mode 100644 CQA_Retriever/widget/README.md diff --git a/CQA_Retriever/README.md b/CQA_Retriever/README.md new file mode 100644 index 0000000..206aa50 --- /dev/null +++ b/CQA_Retriever/README.md @@ -0,0 +1,57 @@ +# CQA Retriever Package + +- [Hub Package Readme](_studio_dependencies/README.md) +- [Widget Readme](widget//_studio_dependencies/README.md) + +# Contributing + +- [Setup Widget Development](widget/README.md) + +## Studio + +Items which are provided to IVA Studio's Workspace and publications are in a +\_studio_dependencies directory. + +Each widget provided in this package has its own directory under widget/ + +## Hub Package + +Hub Package is configured through a few locations. Utilize the structure +below to understand the relationship and purpose of the different +files and locations. + +File content will have instances of {{ Hub Version }} replaced with the +identified version based on git Tag. + +``` +/ +├───\_studio\_dependencies/ +│ ├───ConversationFlowExport/ -- Export of example conversation flows +│ ├───DynamicQuery/ +│ ├───Engagement/ +│ ├───GlobalVariable/ +│ ├───ProxyScript/ +│ ├───README.md -- Hub Readme Document +│ └───HubPackage.json -- Hub Name/Description and Official Verint Package +└───widget/ + ├───README.md -- General Development documentation + ├───{widgetName}/ + │ ├───\_studio\_dependencies/ + │ │ └───README.md -- Widget Readme Document + │ ├───widget.config.json -- Widget Name and Description + │ └───README.md -- Development Notes for specific widget + └───{widgetName2}/ +``` + +### Versioning + +Publication and versioning is determined by repository tags. + +v1.0.0-alpha - This will indicate an alpha build (draft, alpha, beta) are +supported and anything else will result in a draft. + +The published commit will be stored and used to compare with the new tag. The semantic version change will determine the increment type (MAJOR, MINOR, PATCH). + +If the working directory has changes (is dirty), then it will publish as a draft-PATCH. + +If publishing a version tag v1.0.0 then it will be marked as published. diff --git a/CQA_Retriever/_studio_dependencies/ConversationFlowExport/IVA-Solution-Consultants_Peter's-Workspace_Example-RAG-using-CQA_f47dad8f-5b7b-48c3-97bc-7bba542c99b5.json b/CQA_Retriever/_studio_dependencies/ConversationFlowExport/IVA-Solution-Consultants_Peter's-Workspace_Example-RAG-using-CQA_f47dad8f-5b7b-48c3-97bc-7bba542c99b5.json new file mode 100644 index 0000000..8092f8e --- /dev/null +++ b/CQA_Retriever/_studio_dependencies/ConversationFlowExport/IVA-Solution-Consultants_Peter's-Workspace_Example-RAG-using-CQA_f47dad8f-5b7b-48c3-97bc-7bba542c99b5.json @@ -0,0 +1 @@ +{"file":{"content":{"intents":[{"_id":"4d5df90a-66f1-48b1-9762-7d76c888d153-NjgzMTEwY2JkMDM3ZjMzNGIyZTY5MmRk","workspaceId":"4d5df90a-66f1-48b1-9762-7d76c888d153-NjdiY2E4NjIyMTAwNzE2MjdkMzJlZjEy","organizationId":"4d5df90a-66f1-48b1-9762-7d76c888d153-NjRjMmJhMGQ5NDdkMGM3OWY0YjlkN2Ey","modelId":"4d5df90a-66f1-48b1-9762-7d76c888d153-NjdiZTNmNTQwYjY3YmViM2ZkZWM1Mjkw","label":"CQA_Retriever Intent","keyPhrases":[],"enabled":true,"relevant":true,"metadata":{"createdBy":"4d5df90a-66f1-48b1-9762-7d76c888d153-NjRiYWMwNTkyNGE5YzU3ODNlZjc0MjRl","createdAt":1748046013108,"_vc":{"branch":"current","operation":"update","_id":"4d5df90a-66f1-48b1-9762-7d76c888d153-NjgzMTEwYmRkMDM3ZjMzNGIyZTY5MmQ5","lts_id":"4d5df90a-66f1-48b1-9762-7d76c888d153-NjgzMWZiMTY4YzllYjA3MzRjNGM1N2Zk","commitId":"4d5df90a-66f1-48b1-9762-7d76c888d153-NjgzMWZiMTY1YTMyZjIwODMzNDg2NTQz"},"updatedBy":"4d5df90a-66f1-48b1-9762-7d76c888d153-NjRiYWMwNTkyNGE5YzU3ODNlZjc0MjRl","updatedAt":1748106006062}}],"alternates":[],"entities":[],"entityValues":[],"conversationFlows":[{"_id":"4d5df90a-66f1-48b1-9762-7d76c888d153-NjgzMTExMjJkMDM3ZjMzNGIyZTY5MmRm","workspaceId":"4d5df90a-66f1-48b1-9762-7d76c888d153-NjdiY2E4NjIyMTAwNzE2MjdkMzJlZjEy","organizationId":"4d5df90a-66f1-48b1-9762-7d76c888d153-NjRjMmJhMGQ5NDdkMGM3OWY0YjlkN2Ey","modelId":"4d5df90a-66f1-48b1-9762-7d76c888d153-NjdiZTNmNTQwYjY3YmViM2ZkZWM1Mjkw","name":"Example Flow: RAG with CQA","description":"No description","content":{"c403bb8e-93cf-4f7f-aab1-63b94cce8938":{"_id":"6827a5ec0de34003aa0fb403","componentId":"879c32c0-6080-45b7-a694-f1c445c89cde","originalComponentId":"231d1bd0-f185-40c0-b387-8a03b893002f","type":"seed","icon":"fas fa-question","position":{"left":0,"top":0},"name":"CQA","readme":"# CQA Actions Functionality\r\n\r\nThis package contain a widget, a global variable for CQA API connections (**CQAsettings**) and a Global Variable Function (**CQA**) for performing actions against the API.\r\n\r\n\r\n## Setup\r\nThe widget is multi-functional and provides the configuration based on type of data the conversation flow is looking to return. A detailed explanation of each function is below. The widget form groups items needed as **Inputs** to the API call and **Outputs**. Required fields are marked to help make sure necessary data is available for the API. ==At this time there is not any form validation and prevention if data is incorrect or missing.== Each function has a success and failure state. These can be mapped to **Continue to next in group** or **Exit through output**. \r\n\r\n### Generate Answer\r\nThis function will take in the the sources to use to answer the question and return the details of the engine's output(answers), if the instance was processed successfully","description":"CQA","endpoint":"https://console.ivastudio.verint.live/Widget/get/651dbf1f1042401ffa798837_66291d56770103043971bf0f_879c32c0-6080-45b7-a694-f1c445c89cde.js","version":{"major":0,"minor":0,"patch":1},"committish":null,"metadata":{"createdBy":"64bac05924a9c5783ef7424e","createdAt":1747428844511},"id":"c403bb8e-93cf-4f7f-aab1-63b94cce8938","groupId":"790b563c-e950-466a-96a4-73ea5bb592fa","data":{"code":"\n\t\t\t\tlet exitState = 0;\n\t\t\t\t(async () => {\n\t\t\t\t\t\n\t\t\t\t\t\t let data = await CQA().getanswers(conversationData.cqa_source);\n conversationData.cqa_response = data;\n\t\t\t\t\t\t\n\t\t\t\t})().catch((e) => {\n\t\t\t\t\tconsole.log(e.message);\n\t\t\t\t\texitState = 2;\n\t\t\t\t}).finally(() => {\n\t\t\t\t\tswitch(exitState){\n\t\t\t\t\t\tcase 0:\n\t\t\t\t\t\t\tif(\"next()\" === \"next()\") {\n\t\t\t\t\t\t\t\tnext();\n\t\t\t\t\t\t\t} else {\n\t\t\t\t\t\t\t\tnext(exitState);\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\tbreak;\n\t\t\t\t\t\tcase 1:\n\t\t\t\t\t\t\tif(\"next()\" === \"next()\") {\n\t\t\t\t\t\t\t\tnext();\n\t\t\t\t\t\t\t} else {\n\t\t\t\t\t\t\t\tnext(exitState);\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\tbreak;\n\t\t\t\t\t\tcase 2:\n\t\t\t\t\t\t\tif(\"next()\" === \"next()\") {\n\t\t\t\t\t\t\t\tnext();\n\t\t\t\t\t\t\t} else {\n\t\t\t\t\t\t\t\tnext(exitState);\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\tbreak;\n\t\t\t\t\t}\n\t\t\t\t});\n\t\t\t\t","ports":[null,null,null],"getSource":"cqa_source","selectedEndpointLabel":"Generate Answer","getcqaresp":"cqa_response","formValid":true}},"4abeb76e-b0d7-4507-a2e9-0113c623b976":{"_id":"67be46980b67beb3fdec52bd","componentId":"b54cfa4e-e681-4c49-a824-0d9f3718d7cd","originalComponentId":"d9db6b1e-e3d5-40f2-a618-7684a28adfc0","type":"seed","icon":"paragraph","position":{"left":0,"top":0},"name":"Multi-Channel Reply","readme":"# Multi-Channel Reply Widget\r\n\r\nThis widget allows you to craft responses with rich text features\r\nusing a user-friendly WYSIWYG editor.\r\n\r\n## What's New (06/20/2024)\r\n- Added option to exit through port\r\n\r\n## Supported Channels\r\n\r\n- Plain text (fallback)\r\n- Web/HTML\r\n- Facebook\r\n- Microsoft Teams\r\n- Twitter\r\n- WhatsApp\r\n- SMS\r\n\r\n### All Channels\r\n\r\nYou can define multiple messages within a single reply. Use the **Add Content**\r\nbutton to add additional messages or other types of content (depending on the\r\nselected channel.)\r\n\r\nYou can include **inline conversation variables** in your response\r\nby including a variable name surrounded by curly brackets,\r\nsuch as `{variableName}`. Object paths and arrays are also supported, e.g. `{path.to.thing[3]}`.\r\n\r\nA randomizer feature is also included for rich text and plain text replies.\r\nYou can define multiple **alternative responses** for each message,\r\none of which will be selected at random each time the response is hit in the agent.\r\n\r\nYou may also create replies for multiple languages.\r\nUse the dropdown at the top of the form to select the language currently being edited.\r\nAll languages will share the same content cards for every channel.\r\n\r\n### Web Channel\r\n\r\nSupported features include:\r\n\r\n- **Bold**\r\n- _Italic_\r\n- Bulleted lists\r\n- Numbered lists\r\n- Hyperlinks\r\n- Images\r\n- Block quotes\r\n- Quick replies\r\n- Undo/redo\r\n- Copy/paste\r\n\r\nThe editor saves the response in HTML format. If desired,\r\nyou can view and edit the raw HTML markup by toggling the switch\r\nfrom **Rich** to **Raw**.\r\n\r\n### Facebook Channel\r\n\r\nSupported features include:\r\n\r\n- **Bold**\r\n- _Italic_\r\n- ~~Strikethrough~~\r\n- Carousel\r\n- Image replies\r\n- Quick replies\r\n- Undo/redo\r\n- Copy/paste\r\n\r\nThe editor saves the response in Facebook message format. If desired,\r\nyou can view and edit the raw markup by toggling the switch\r\nfrom **Rich** to **Raw**.\r\n\r\n### Microsoft Teams Channel\r\n\r\nSupported features include:\r\n\r\n- **Bold**\r\n- _Italic_\r\n- Bulleted lists\r\n- Numbered lists\r\n- Hyperlinks\r\n- Block quotes\r\n- Quick replies\r\n- Cards\r\n - Hero Card\r\n - Thumbnail Card\r\n- Undo/redo\r\n- Copy/paste\r\n\r\nThe editor saves the response in Markdown format. If desired,\r\nyou can view and edit the raw Markdown markup by toggling the switch\r\nfrom **Rich** to **Raw**.\r\n\r\n### Twitter Channel\r\n\r\nSupported features include:\r\n\r\n- Plain text replies\r\n- Quick Reply messages\r\n- Calls to Action\r\n\r\n### WhatsApp Channel\r\n\r\nSupported features include:\r\n\r\n- **Bold**\r\n- _Italic_\r\n- ~~Strikethrough~~\r\n- Quick Replies\r\n- Menu Replies\r\n- Media Replies\r\n - Image\r\n - Video\r\n - Audio\r\n - Document (txt, PDF, Word, Excel, Powerpoint)\r\n- Undo/redo\r\n- Copy/paste\r\n\r\nThe editor saves the response in WhatsApp message format. If desired,\r\nyou can view and edit the raw markup by toggling the switch\r\nfrom **Rich** to **Raw**.\r\n","description":"Craft rich text replies for multiple different channels","endpoint":"https://console.ivastudio.verint.live/Widget/get/64e3ca2df0c99192d946ab9e_655d44bd75c5d32ee18263ca_b54cfa4e-e681-4c49-a824-0d9f3718d7cd.js","version":{"major":0,"minor":1,"patch":22},"metadata":{"createdBy":"64bac05924a9c5783ef7424e","createdAt":1740523160173},"id":"4abeb76e-b0d7-4507-a2e9-0113c623b976","groupId":"790b563c-e950-466a-96a4-73ea5bb592fa","data":{"code":"// Auto-generated code. Do not modify. [Multi-Channel Reply Widget]\n(async () => {\n\tconst channels = {\"text\":{\"en\":[[\"\\nAnswer: {cqa_response.answer}\\nEvidence Text: {cqa_response.evidence.text}\\nEvidence Source ID: {cqa_response.evidence.source_id}\"]]},\"web\":{\"en\":[[\"

Answer:

{cqa_response.answer}

Evidence Text:

{cqa_response.evidence.text}

Evidence Source ID:

{cqa_response.evidence.source_id}

\"]]},\"voice\":{\"en\":[[\"{cqa_response.answer}\"],[\"How else can I help you?\"]]}};\n\tconst channelOptions = {};\n\tconst channelName = (req.body?.metadata?.channel ?? req.body?.channel ?? \"web\").toLowerCase();\n\tconst languages = channels[channelName] ?? channels.text ?? channels.web;\n\tconst lang = (req.body?.metadata?.lang ?? req.body?.lang ?? \"en\").slice(0, 2);\n\tconst replies = languages?.[lang] ?? [[\"\"]];\n\treplies.forEach(replySet => {\n\t\tfunction replaceVariables(str) {\n\t\t\treturn str.replace(/{([^}]+)}/g, (match, key) => (new Function(\"x\", \"return x.\" + key))(conversationData));\n\t\t}\n\t\tlet reply = replySet.length > 1 ? replySet[Math.floor(Math.random() * replySet.length)] : replySet[0];\n\t\tif (typeof reply === 'object') {\n\t\t\tfunction replaceValues(obj) {\n\t\t\t\tif (Array.isArray(obj)) {\n\t\t\t\t\tfor (let i = 0; i < obj.length; i++) {\n\t\t\t\t\t\tif (typeof obj[i] === \"object\") {\n\t\t\t\t\t\t\treplaceValues(obj[i]);\n\t\t\t\t\t\t}\n\t\t\t\t\t\telse if (typeof obj[i] === \"string\") {\n\t\t\t\t\t\t\tobj[i] = replaceVariables(obj[i]);\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\telse {\n\t\t\t\t\tfor (const key in obj){\n\t\t\t\t\t\tif (typeof obj[key] === \"object\") {\n\t\t\t\t\t\t\treplaceValues(obj[key]);\n\t\t\t\t\t\t}\n\t\t\t\t\t\telse if (typeof obj[key] === \"string\") {\n\t\t\t\t\t\t\tobj[key] = replaceVariables(obj[key]);\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t\treplaceValues(reply);\n\t\t}\n\t\telse {\n\t\t\treply = replaceVariables(reply);\n\t\t}\n\t\trecognizedObject.answers.push(reply);\n\t});\n\tconst options = channelOptions[channelName]?.[lang];\n\tif (options && options.length > 0) {\n\t\tif (!(\"options\" in recognizedObject)) recognizedObject.options = [];\n\t\trecognizedObject.options.push(...options);\n\t}\n\tnext();\n})().catch((e) => {\n\trecognizedObject.errorInfo = {\n\t\terror: e.message,\n\t\tstack: e.stack\n\t}\n\tif (e.toJSON) recognizedObject.errorInfo = e.toJSON();\n if (e.response) recognizedObject.errorInfo.response = e.response.data;\n\tconsole.log(recognizedObject.errorInfo);\n\tend();\n});","ports":[null],"valid":true,"activeChannels":{"text":[{"type":"text","languages":{"en":[{"value":"\nAnswer: {cqa_response.answer}\nEvidence Text: {cqa_response.evidence.text}\nEvidence Source ID: {cqa_response.evidence.source_id}","valid":true}]}}],"web":[{"type":"text","languages":{"en":[{"value":"

Answer:

{cqa_response.answer}

Evidence Text:

{cqa_response.evidence.text}

Evidence Source ID:

{cqa_response.evidence.source_id}

","valid":true}]}}],"voice":[{"type":"text","languages":{"en":[{"value":"{cqa_response.answer}","valid":true}]}},{"type":"text","languages":{"en":[{"value":"How else can I help you?","valid":true}]}}]},"enablePort":false,"updateKey":883289263}},"790b563c-e950-466a-96a4-73ea5bb592fa":{"type":"group","componentId":"group","icon":"layer-group","position":{"top":5336,"left":5798},"name":"Group","description":"Useful for grouping widgets together","id":"790b563c-e950-466a-96a4-73ea5bb592fa","data":{"steps":["c403bb8e-93cf-4f7f-aab1-63b94cce8938","4abeb76e-b0d7-4507-a2e9-0113c623b976"],"name":"Augmented Generation"}},"77b8723e-8451-4779-88e7-299e4800e5bb":{"_id":"67be46980b67beb3fdec52bf","componentId":"f954900c-e566-4382-847e-2ff8a07a2f77","originalComponentId":"31910655-cc5a-45e9-b157-2927412e74b9","type":"seed","icon":"puzzle-piece","position":{"left":0,"top":0},"name":"Slot","readme":"# Slot\r\nThe data collected can be:\r\n\r\n- Entire users input\r\n- Extractions from a system entity like dates, etc.\r\n- Extractions from a user specified regex entity\r\n- String entity option that matched\r\n \r\nBy default there will be a single attempt to collect this data on the next user input and presumably a previous widget provided a reply to the user asking for this data.\r\n \r\nOptionally the widget can do the following:\r\n \r\n- Provide the initial prompt itself with simple text\r\n- Re-prompt with text and a specified number of times in cases where the user’s answer did not result in collecting data\r\n- Attempt to collect the data from the current user input without needing to prompt and collect in the next turn of the conversation","description":"It can be used to collect data from a user input and store it in a specified conversation variable.","endpoint":"https://console.ivastudio.verint.live/Widget/get/64e3ca2df0c99192d946ab9e_664c4ef98d813c3086d99a05_f954900c-e566-4382-847e-2ff8a07a2f77.js","version":{"major":0,"minor":0,"patch":8},"metadata":{"createdBy":"64bac05924a9c5783ef7424e","createdAt":1740523160238},"id":"77b8723e-8451-4779-88e7-299e4800e5bb","groupId":"43d4045f-1c43-466c-8042-ec2fd968c1f3","data":{"code":"(async () => {\nconsole.log = conversationData?.metadata?.verbose? console.log : () => {};\nconst slot = {\n \"cpyqFQRv1OHalA15WChQF\": {\n \"id\": \"cpyqFQRv1OHalA15WChQF\",\n \"pristine\": true,\n \"promptText\": \"State your question for the RAG Example\",\n \"counters\": {\n \"cqa_question\": 0\n },\n \"props\": [\n {\n \"editor\": \"\",\n \"validator\": \"// \\\"value\\\" is equal to extracted entity value\\n// return true or false to accept or reject value respectfully\\nreturn true;\",\n \"promptText\": \"Question\",\n \"selectedEntity\": [\n \"user_input\"\n ],\n \"dataVariable\": \"cqa_question\"\n }\n ],\n \"allowedIntents\": []\n }\n};\nconversationData.slot = _.merge({}, slot, conversationData.slot);\nrecognizedObject.slot = conversationData.slot;\nconst validator = (name, value) => {\n\tconst get = () => {\n\t\tswitch (name) {\n \t\t\tcase 'cqa_question':\nreturn (value) => {\n// \"value\" is equal to extracted entity value\n// return true or false to accept or reject value respectfully\nreturn true;\n\t\t\t\t};\n\t\t\tdefault:\nreturn (value) => {\n\treturn true;\n};\n\t\t}\n\t};\nreturn get()(value);\n};\n\nconst editor = (name) => {\n\tconst get = () => {\n\t\tswitch (name) {\n \t\t\tcase 'cqa_question':\nreturn (value) => {\n\n\t\t\t\t};\n\t\t\tdefault:\nreturn (value) => {\n};\n\t\t}\n\t};\nreturn get()();\n};\n\nconst replaceVariables = (template, data) => {\n\treturn template.replace(/\\{([\\w.]+)\\}/g, (match, variable) => {\n\t\tconst value = variable.split('.').reduce((obj, key) => (obj && obj[key] !== undefined) ? obj[key] : undefined, data);\n\t\treturn value !== undefined ? value : match;\n\t});\n}\n\nconst getExpectedPhrasesFor = (entity) => {\n\tconst systemPhrases = [];\n\tif (entity === \"number\" || entity === \"duration\") {\n\t\tsystemPhrases.push(\"$OOV_CLASS_DIGIT_SEQUENCE\");\n\t\tsystemPhrases.push(\"$OPERAND\");\n\t}\n\tif (entity === \"phonenumber\") {\n\t\tsystemPhrases.push(\"$FULLPHONENUM\");\n\t\tsystemPhrases.push(\"$OOV_CLASS_FULLPHONENUM\");\n\t}\n\tif (entity === \"ordinal\") {\n\t\tsystemPhrases.push(\"$OOV_CLASS_ORDINAL\");\n\t}\n\tif (entity === \"time\") {\n\t\tsystemPhrases.push(\"$TIME\");\n\t}\n\tif (entity === \"date\" || entity === \"date_past\") {\n\t\tsystemPhrases.push(\"$MONTH\");\n\t\tsystemPhrases.push(\"$DAY\");\n\t\tsystemPhrases.push(\"$YEAR\");\n\t\tsystemPhrases.push(\"$MONTH $DAY\");\n\t\tsystemPhrases.push(\"$MONTH $DAY $YEAR\");\n\t}\n\tif (entity === \"currency\") {\n\t\tsystemPhrases.push(\"$MONEY\");\n\t}\n\treturn systemPhrases;\n};\n\nconst process = (slot) => {\n\tconst entities = recognizedObject.nerResults.entities.map((entity, index) => {\n\t\tconst entities = [];\n\t\tconst clone = _.merge({}, entity);\n\t\tentities.push(clone);\n\t\tif (clone.entity == \"daterange\" && clone.resolution.type === 'interval') {\n\t\t\tentities.push({\n\t\t\t\t...clone,\n\t\t\t\t'entity': 'date',\n\t\t\t\t\"rawEntity\": \"datetimeV2.date\",\n\t\t\t\t\"resolution\": {\n\t\t\t\t\t\"type\": \"interval\",\n\t\t\t\t\t\"timex\": clone.resolution.timex,\n\t\t\t\t\t\"strPastValue\": clone.resolution.strPastStartValue,\n\t\t\t\t\t\"pastDate\": clone.resolution.pastStartDate,\n\t\t\t\t\t\"strFutureValue\": clone.resolution.strFutureStartValue,\n\t\t\t\t\t\"futureDate\": clone.resolution.futureStartDate\n\t\t\t\t}\n\t\t\t});\n\t\t\tentities.push({\n\t\t\t\t...clone,\n\t\t\t\t'entity': 'date',\n\t\t\t\t\"rawEntity\": \"datetimeV2.date\",\n\t\t\t\t\"resolution\": {\n\t\t\t\t\t\"type\": \"interval\",\n\t\t\t\t\t\"timex\": clone.resolution.timex,\n\t\t\t\t\t\"strPastValue\": clone.resolution.strPastEndValue,\n\t\t\t\t\t\"pastDate\": clone.resolution.pastEndDate,\n\t\t\t\t\t\"strFutureValue\": clone.resolution.strFutureEndValue,\n\t\t\t\t\t\"futureDate\": clone.resolution.futureEndDate\n\t\t\t\t}\n\t\t\t})\n\t\t} else if (clone.entity == \"date\" && clone.resolution.strValue) {\n\t\t\tif (clone.resolution.strValue !== \"not resolved\") {\t\t\t\n\t\t\t\tif (datefns.isPast(new Date(clone.resolution.strValue))) {\n\t\t\t\t\tclone.resolution.strPastValue = clone.resolution.strValue;\n\t\t\t\t\tclone.resolution.pastDate = new Date(clone.resolution.strValue).toISOString();\n\t\t\t\t} else {\n\t\t\t\t\tclone.resolution.strFutureValue = clone.resolution.strValue;\n\t\t\t\t\tclone.resolution.futureDate = new Date(clone.resolution.strValue).toISOString();\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\treturn [];\n\t\t\t}\n\t\t\tdelete clone.resolution.strValue;\n\t\t} else if (clone.entity === \"time\") {\n\t\t\tif (clone?.resolution?.values) {\n\t\t\t\tclone.resolution = clone.resolution.values[0];\n\t\t\t}\n\t\t}\n\t\tentities.sort((a, b) => b.entity.length - a.entity.length);\n\t\treturn entities;\n\t}).flat();\n\tconsole.log({ entities });\n\tslot.props.filter((prop) => {\n\t\treturn _.isNil(prop.value) || (_.isArray(prop.value) && _.isEmpty(prop.value));\n\t}).forEach((prop, idx, props) => {\n\t\tif (prop.selectedEntity.includes(\"user_input\")) {\n\t\t\tprops[idx].value = recognizedObject.input;\n\t\t\tif (validator(prop.dataVariable, props[idx].value)) {\n\t\t\t} else {\n\t\t\t\tdelete props[idx].value;\n\t\t\t}\n\t\t} else {\n\t\t\tconst match = (entity) => {\n\t\t\t\treturn prop.selectedEntity.some((selected) => {\n\t\t\t\t\tif (selected === \"date_past\" && entity.resolution?.strPastValue) return true;\n\t\t\t\t\tif (['regex', 'enum'].includes(entity.type)) return entity.entity.includes(selected);\n\t\t\t\t\treturn entity.entity === selected;\n\t\t\t\t});\n\t\t\t};\n\t\t\tif (prop.listSwitch) {\n\t\t\t\tprops[idx].value = entities.filter(match);\n\t\t\t\tprops[idx].value.forEach((entity, idx, array) => {\n\t\t\t\t\tarray[idx] = entity.resolution?.value ?? entity.resolution?.strValue ?? (prop.selectedEntity.includes(\"date_past\") ? entity.resolution?.strPastValue : entity.resolution?.strFutureValue) ?? (prop.selectedEntity.includes(\"date_past\") ? entity.resolution?.strFutureValue : entity.resolution?.strPastValue) ?? entity.option ?? entity.sourceText;\n\t\t\t\t});\n\t\t\t\tif (validator(prop.dataVariable, props[idx].value)) {\n\t\t\t\t} else {\n\t\t\t\t\tdelete props[idx].value;\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tconst edx = entities.findIndex(match);\n\t\t\t\tif (edx > -1) {\n\t\t\t\t\tprops[idx].value = entities[edx];\n\t\t\t\t\tprops[idx].value = props[idx].value.resolution?.value ?? props[idx].value.resolution?.strValue ?? (prop.selectedEntity.includes(\"date_past\") ? props[idx].value.resolution?.strPastValue : props[idx].value.resolution?.strFutureValue) ?? (prop.selectedEntity.includes(\"date_past\") ? props[idx].value.resolution?.strFutureValue : props[idx].value.resolution?.strPastValue) ?? props[idx].value.option ?? props[idx].value.sourceText;\n\t\t\t\t\tif (validator(prop.dataVariable, props[idx].value)) {\n\t\t\t\t\t} else {\n\t\t\t\t\t\tdelete props[idx].value;\n\t\t\t\t\t}\n\t\t\t\t\tentities.splice(edx, 1);\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t});\n};\nconst reset = (slot) => {\n\tdelete conversationData.slot['cpyqFQRv1OHalA15WChQF'];\n};\nconst answer = (slot) => {\n\tconst remainders = slot.props.filter((prop) => {\n\t\treturn _.isNil(prop.value) || (_.isArray(prop.value) && _.isEmpty(prop.value));\n\t}).length;\n\tif (remainders) {\n\t\tconst { promptText, dataVariable, selectedEntity } = slot.props.find((prop) => {\n\t\t\treturn _.isNil(prop.value) || (_.isArray(prop.value) && _.isEmpty(prop.value));\n\t\t});\n\t\tconst allowIntent = () => {\n\t\t\tdelete recognizedObject.match;\n\t\t\tconst [intent] = recognizedObject.classificationResults;\n\t\t\tif (intent) {\n\t\t\t\tconst allowedIntent = slot.allowedIntents.find((allowedIntent) => {\n\t\t\t\t\treturn allowedIntent._id === intent._id || allowedIntent._id === intent?.metadata?._vc?._id || allowedIntent?.metadata?._vc?._id === intent._id;\n\t\t\t\t});\n\t\t\t\tif (allowedIntent) recognizedObject.match = intent;\n\t\t\t}\n\t\t\tif (recognizedObject.match) {\n\t\t\t\trecognizedObject.pathName = `Slot(${slot.id}[AllowedIntent(${recognizedObject.match.label})])`;\n\t\t\t\treturn true;\n\t\t\t}\n\t\t}\n\t\tif (slot.pristine && slot.promptText) {\n\t\t\tslot.pristine = false;\n\t\t\trecognizedObject.answers.push(slot.promptText);\n\t\t\trecognizedObject.pathName = `Slot(${slot.id})`;\n\t\t\tpause();\n\t\t} else if (slot.counters[dataVariable] < 2) {\n\t\t\tslot.pristine = false;\n\t\t\tif (allowIntent()) {\n\t\t\t\treset(slot);\n\t\t\t\tnext(2);\n\t\t\t} else {\n\t\t\t\tif ((false && slot.counters[dataVariable] === 0) || (slot.counters[dataVariable] === 0 && !slot.promptText)) {} else {\n\t\t\t\t\trecognizedObject.classificationResults = [];\n\t\t\t\t}\n\t\t\t\trecognizedObject.answers.push(promptText);\n\t\t\t\trecognizedObject.expectedPhrases = [];\n\t\t\t\tselectedEntity.forEach((entity) => {\n\t\t\t\t\trecognizedObject.expectedPhrases.push(...getExpectedPhrasesFor(entity));\n\t\t\t\t});\n\t\t\t\teditor(dataVariable);\n\t\t\t\trecognizedObject.pathName = `Slot(${slot.id}[${dataVariable}])`;\n\t\t\t\tslot.counters[dataVariable]++;\n\t\t\t\tpause();\n\t\t\t}\n\t\t} else {\n\t\t\tif (allowIntent()) {\n\t\t\t\treset(slot);\n\t\t\t\tnext(2);\n\t\t\t} else {\n\t\t\t\tif (slot.counters[dataVariable] > 0) {\n\t\t\t\t\trecognizedObject.classificationResults = [];\n\t\t\t\t}\n\t\t\t\treset(slot);\n\t\t\t\tnext();\n\t\t\t}\n\t\t}\n\t} else {\n\t\tif (false && Object.values(slot.counters).reduce((p, c) => p + c, 0) === 0) {} else {\n\t\t\trecognizedObject.classificationResults = [];\n\t\t}\n\t\tslot.props.forEach((prop) => {\n\t\t\tconversationData[prop.dataVariable] = prop.value;\n\t\t});\n\t\treset(slot);\n\t\tnext();\n\t}\n};\nconst init = (id) => {\n\tconst slot = conversationData.slot[id];\n\tif (slot.promptText) {\n\t\tslot.promptText = replaceVariables(slot.promptText, conversationData);\n\t}\n\tslot.props.forEach((prop, index, array) => {\n\t\tarray[index].promptText = replaceVariables(prop.promptText, conversationData);\n\t});\n\tif (slot.pristine === false) {\n\t\tprocess(slot);\n\t} else {\n\t\tif (false) {\n\t\t\tprocess(slot);\n\t\t}\n\t}\n\tanswer(slot);\n};\ninit('cpyqFQRv1OHalA15WChQF');\n})().catch((e) => {\n\trecognizedObject.errorInfo = {\n\t\terror: e.message,\n\t\tstack: e.stack\n\t}\n\tif (e.toJSON) recognizedObject.errorInfo = e.toJSON();\n if (e.response) recognizedObject.errorInfo.response = e.response.data;\n\tconsole.log(recognizedObject.errorInfo);\n\tend();\n});\n","ports":[null,null,null],"formValid":true,"sufix":"cpyqFQRv1OHalA15WChQF","promptSwitch":true,"promptText":"State your question for the RAG Example","extractions":[{"editor":"","validator":"// \"value\" is equal to extracted entity value\n// return true or false to accept or reject value respectfully\nreturn true;","promptText":"Question","selectedEntity":["user_input"],"dataVariable":"cqa_question"}],"allowedIntents":[],"propagation":false,"promptCount":2,"selectedCounterAction":"next()","selectedFilledAction":"next()","selectedSource":"conversationData"}},"9ad68eeb-b287-4248-893b-050364a6a4dd":{"type":"seed","componentId":"seed","icon":"code","position":{"left":0,"top":0},"name":"Code","description":"Allows custom code to execute where placed","id":"9ad68eeb-b287-4248-893b-050364a6a4dd","groupId":"43d4045f-1c43-466c-8042-ec2fd968c1f3","data":{"code":"(async () => {\n console.log(`CQA Retrieval: ${conversationData.cqa_question}`)\n conversationData.cqa_source = await CQA_Retriever().retrieve(conversationData.cqa_question);\n})()\n .catch((error) => {\n console.log(error.message)\n recognizedObject.answers.push(error.message)\n recognizedObject.errorInfo = {\n ...recognizedObject.errorInfo,\n label: {\n data: error.toJSON ? error.toJSON() : {},\n message: error.message,\n },\n }\n })\n .finally(() => {\n next(0)\n })","ports":[{"id":"c403bb8e-93cf-4f7f-aab1-63b94cce8938"}],"name":"CQA Retrieval"}},"43d4045f-1c43-466c-8042-ec2fd968c1f3":{"type":"group","componentId":"group","icon":"layer-group","position":{"top":5271,"left":5457},"name":"Group","description":"Useful for grouping widgets together","id":"43d4045f-1c43-466c-8042-ec2fd968c1f3","data":{"steps":["77b8723e-8451-4779-88e7-299e4800e5bb","9ad68eeb-b287-4248-893b-050364a6a4dd"],"name":"Retrieval\n"}},"9679a4c9-d63a-4d01-a0ba-6d652fec81d3":{"type":"intent","componentId":"intent","icon":"ear-listen","position":{"left":5105,"top":5276},"name":"Intent","description":"Select intents as flow triggers","id":"9679a4c9-d63a-4d01-a0ba-6d652fec81d3","data":{"intentId":"683110bdd037f334b2e692d9","ports":[{"id":"77b8723e-8451-4779-88e7-299e4800e5bb"}]}}},"global":false,"metadata":{"createdBy":"4d5df90a-66f1-48b1-9762-7d76c888d153-NjRiYWMwNTkyNGE5YzU3ODNlZjc0MjRl","createdAt":1748045910404,"_vc":{"branch":"current","operation":"update","_id":"4d5df90a-66f1-48b1-9762-7d76c888d153-NjgzMTEwNTZkMDM3ZjMzNGIyZTY5MmQ3","lts_id":"4d5df90a-66f1-48b1-9762-7d76c888d153-NjgzMWZiMTY4YzllYjA3MzRjNGM1ODAy","commitId":"4d5df90a-66f1-48b1-9762-7d76c888d153-NjgzMWZiMTY1YTMyZjIwODMzNDg2NTQz"},"updatedBy":"4d5df90a-66f1-48b1-9762-7d76c888d153-NjRiYWMwNTkyNGE5YzU3ODNlZjc0MjRl","updatedAt":1748106006072},"dependsOn":[{"_id":"4d5df90a-66f1-48b1-9762-7d76c888d153-NjgzMTEwYmRkMDM3ZjMzNGIyZTY5MmQ5","_vc_latest":{"_id":"4d5df90a-66f1-48b1-9762-7d76c888d153-NjgzMTEwY2JkMDM3ZjMzNGIyZTY5MmRk","workspaceId":"4d5df90a-66f1-48b1-9762-7d76c888d153-NjdiY2E4NjIyMTAwNzE2MjdkMzJlZjEy","organizationId":"4d5df90a-66f1-48b1-9762-7d76c888d153-NjRjMmJhMGQ5NDdkMGM3OWY0YjlkN2Ey","modelId":"4d5df90a-66f1-48b1-9762-7d76c888d153-NjdiZTNmNTQwYjY3YmViM2ZkZWM1Mjkw","label":"CQA_Retriever Intent","keyPhrases":[],"enabled":true,"relevant":true,"publication":null,"metadata":{"createdBy":"4d5df90a-66f1-48b1-9762-7d76c888d153-NjRiYWMwNTkyNGE5YzU3ODNlZjc0MjRl","createdAt":1748046013108,"_vc":{"branch":"current","operation":"update","_id":"4d5df90a-66f1-48b1-9762-7d76c888d153-NjgzMTEwYmRkMDM3ZjMzNGIyZTY5MmQ5","lts_id":"4d5df90a-66f1-48b1-9762-7d76c888d153-NjgzMWZiMTY4YzllYjA3MzRjNGM1N2Zk","commitId":"4d5df90a-66f1-48b1-9762-7d76c888d153-NjgzMWZiMTY1YTMyZjIwODMzNDg2NTQz"},"updatedBy":"4d5df90a-66f1-48b1-9762-7d76c888d153-NjRiYWMwNTkyNGE5YzU3ODNlZjc0MjRl","updatedAt":1748106006062}}}]}],"globalVariables":[],"proxyScripts":[],"dynamicQueries":[],"llmConfigs":[],"prompts":[],"widgets":[],"engagements":[]},"authorId":"4d5df90a-66f1-48b1-9762-7d76c888d153-NjRiYWMwNTkyNGE5YzU3ODNlZjc0MjRl","description":"This package uses Azure AI Search with CQA as an example RAG flow","readme":"# CQA Retrieval Package","entities":{"Widget":[],"Dashboard":[],"DynamicQuery":[],"GlobalVariable":[],"Engagement":[],"ProxyScript":[],"Alternate":[],"ConversationFlow":["4d5df90a-66f1-48b1-9762-7d76c888d153-NjgzMTExMjJkMDM3ZjMzNGIyZTY5MmRm"],"EntityValue":[],"Entity":[],"Intent":["4d5df90a-66f1-48b1-9762-7d76c888d153-NjgzMTEwY2JkMDM3ZjMzNGIyZTY5MmRk"],"LlmConfig":[],"Prompt":[]},"name":"Example RAG using CQA","status":"PENDING","version":{"major":0,"minor":0,"patch":1}},"id":"f9f2a69a505221015d9308df7380ed240a834095360186a947c5b93a5158ac2a"} \ No newline at end of file diff --git a/CQA_Retriever/_studio_dependencies/GlobalVariable/CQA_Retriever.js b/CQA_Retriever/_studio_dependencies/GlobalVariable/CQA_Retriever.js new file mode 100644 index 0000000..7f900ef --- /dev/null +++ b/CQA_Retriever/_studio_dependencies/GlobalVariable/CQA_Retriever.js @@ -0,0 +1,74 @@ +const filter = { + filterExpression: CQA_RetrieverSettings.filterExpression, +}; + +const embedding = new langchain.openai.AzureOpenAIEmbeddings({ + azureOpenAIApiInstanceName: + CQA_RetrieverSettings.azure_openai_api.instance_name, + azureOpenAIApiDeploymentName: + CQA_RetrieverSettings.azure_openai_api.deployment_name, + azureOpenAIApiVersion: CQA_RetrieverSettings.azure_openai_api.version, + azureOpenAIApiKey: CQA_RetrieverSettings.azure_openai_api.key, +}); + +const store = + new langchain.community.vectorstores.azure_aisearch.AzureAISearchVectorStore( + embedding, + { + endpoint: CQA_RetrieverSettings.azure_aisearch.endpoint, + key: CQA_RetrieverSettings.azure_aisearch.key, + indexName: CQA_RetrieverSettings.azure_aisearch.index_name, + search: { + type: langchain.community.vectorstores.azure_aisearch + .AzureAISearchQueryType.SimilarityHybrid, + }, + } + ); + +function getSourceId(document) { + if (document.metadata) { + const mergedMetadata = Object.values(document.metadata).join(""); + const metatDataObj = JSON.parse(mergedMetadata); + + if ("sourceURL" in metatDataObj) { + return metatDataObj.sourceURL; + } + if ("source" in metatDataObj) { + return metatDataObj.source; + } + if ("source_id" in metatDataObj) { + return metatDataObj.source_id; + } + if ("sourceName" in metatDataObj) { + return metatDataObj.sourceName; + } + } else return "no source found"; +} + +return { + async retrieve(query) { + const resultDocuments = await store.similaritySearch(query, 20, filter); + const sources = resultDocuments.map((doc) => ({ + source_id: getSourceId(doc), + text: doc.pageContent, + })); + + const cqaSources = { + instances: [ + { + sources: sources, + question: query, + generate_question: true, + knowledgebase_description: "iva-vector-demo", + extra_guidance: "", + language_code: "en-GB", + }, + ], + }; + + if (CQA_RetrieverSettings.debug) + console.log(JSON.stringify(cqaSources, null, 2)); + + return cqaSources; + }, +}; diff --git a/CQA_Retriever/_studio_dependencies/GlobalVariable/CQA_RetrieverSettings.json b/CQA_Retriever/_studio_dependencies/GlobalVariable/CQA_RetrieverSettings.json new file mode 100644 index 0000000..e96193e --- /dev/null +++ b/CQA_Retriever/_studio_dependencies/GlobalVariable/CQA_RetrieverSettings.json @@ -0,0 +1,16 @@ +{ + "azure_aisearch": { + "endpoint": "https://iva-demo-vector-service.search.windows.net", + "key": "", + "index_name": "iva-vector-demo" + }, + "azure_openai_api": { + "key": "", + "instance_name": "iva-open-ai", + "deployment_name": "text-embedding-3-small", + "embeddings_deployment_name": null, + "version": "2024-08-01-preview" + }, + "filterExpression": "search.in(company, 'Verint')", + "debug": true +} diff --git a/CQA_Retriever/_studio_dependencies/HubPackage.json b/CQA_Retriever/_studio_dependencies/HubPackage.json new file mode 100644 index 0000000..6db9bcb --- /dev/null +++ b/CQA_Retriever/_studio_dependencies/HubPackage.json @@ -0,0 +1,5 @@ +{ + "Name": "CQA Retriever", + "Description": "Example of a RAG retriever using the CQA API", + "OfficialVerintPackage": false +} diff --git a/CQA_Retriever/_studio_dependencies/README.md b/CQA_Retriever/_studio_dependencies/README.md new file mode 100644 index 0000000..fe62999 --- /dev/null +++ b/CQA_Retriever/_studio_dependencies/README.md @@ -0,0 +1,80 @@ +# CQA Retriever + +This package provides an example RAG process using Azure AI Search for Retrieval and the Verint DaVinci Contextual Question Answer (CQA) Service. + +## What's New: Updated [24/05/2025] + +- Initial Release + +## Setup Instructions + +### Prerequisites + +- This package requires the CQA Widget to be installed and configured. + +### Installation + +1. Copy the CQA_Retriever and CQA_RetieverSettings files into **Global Variables** +2. _Optional:_ Import the Example Conversation Flow and Intent + +### Configuration + +#### CQA_RetieverSettings + +Fill out the settings below. + +```json +{ + "azure_aisearch": { + "endpoint": "https://iva-demo-vector-service.search.windows.net", + "key": "", + "index_name": "iva-vector-demo" + }, + "azure_openai_api": { + "key": "", + "instance_name": "iva-open-ai", + "deployment_name": "text-embedding-3-small", + "embeddings_deployment_name": null, + "version": "2024-08-01-preview" + }, + "filterExpression": "search.in(company, 'Verint')", + "debug": true +} +``` + +## Package Content Details + +### CQA_RetieverSettings + +A global variable JSON object with environment-specific settings. See above for details. + +### CQA_Retriever + +A global variable function which handles the logic and API calls used to retrieve documents for Context. + +### Example code block that should be using in your Conversation Flows + +This is included in the Example Conversation Flow if you have imported that. + +```javascript +(async () => { + console.log(`CQA Retrieval: ${conversationData.cqa_question}`); + conversationData.cqa_source = await CQA_Retriever().retrieve( + conversationData.cqa_question + ); +})() + .catch((error) => { + console.log(error.message); + recognizedObject.answers.push(error.message); + recognizedObject.errorInfo = { + ...recognizedObject.errorInfo, + label: { + data: error.toJSON ? error.toJSON() : {}, + message: error.message, + }, + }; + }) + .finally(() => { + next(0); + }); +``` diff --git a/CQA_Retriever/widget/README.md b/CQA_Retriever/widget/README.md new file mode 100644 index 0000000..e62e093 --- /dev/null +++ b/CQA_Retriever/widget/README.md @@ -0,0 +1,7 @@ +# Vue 3 + Vite + +This template should help get you started developing with Vue 3 in Vite. The template uses Vue 3 `