Skip to content Skip to main navigation Skip to footer

How to Use AI to Assist with Checking Entry Data

Please note that before you can use this functionality on your instance, you will need to submit a support ticket so that we can connect to an AI service. Using an AI service will incur additional costs.

Nucleus now supports an integration with Chat GPT, which can be used to review and sense-check answers when the Entry is submitted. The AI will score the answer and allocate a status to the answer, just as Admin users would when checking Entries. The status can be used as a guide for Admin Users to approve the Entry or unsubmit and request an updated answer from the Entrant. At this stage, there is no auto approve/reject feature based on the AI assessment of the answer; however, this technology is at an MVP state, and BMT is happy to work with clients who wish to explore the use of AI within Nucleus.

The Nucleus application of the technology allows Admins to specify a number of AI judges; this refers to how many times the answer will be reviewed by the AI. Each time the AI reviews the answer, it will provide an opinion as to how well the answer meets the criteria provided and a score from 1-10. The scores will be averaged, and the average score used to determine if the question is flagged as Checked OK, Query, or Problem.

Note – There is no limit to how many AI judges will review each answer; however, each judge is an additional AI call which will count towards the number of calls included in your package.

How to Use AI Entry Checking

Once your Chat GPT account has been linked with your Nucleus instance, it is ready for admins to utilise. The AI is applied on a per-question basis and managed in the Question Settings. BMT can provide assistance as to how this can be best utilised to support your Entry processes.

  1. Log into the Admin Interface.
  2. Go to Actions > Question List.
  3. Click Edit on the question you would like AI to check the answers.
  4. Scroll to the section AI Assessment.
  5. Select Yes for AI Assessment Enabled?
  6. Specify the Number of Judges who will assess each answer.
  7. Provide a Prompt – In this field, clearly describe the nature of the question and specify the requirements and expectations for the answer. Detail what the AI needs to understand about the topic and the question’s context. Explain how the AI should assess the response, including the criteria it should use to evaluate the effectiveness, accuracy, and relevance of the answer. Ensure the instructions are precise and concise to guide the AI in providing a thorough and relevant assessment.
  8. Provide what constitutes a Good Answer – Use this field to detail why the response is effective. Highlight strengths such as relevance, clarity, strategic insight, and evidence of outcomes. Emphasise the demonstration of critical skills like leadership, problem-solving, and communication. Ensure examples are specific and directly tied to the criteria. Do not simply provide an example of a Good Answer, but you can provide examples as part of the detail.
  9. Provide what constitutes a Bad Answer – Use this field to identify weaknesses in the response. Note any vagueness, irrelevance, poor structure, or lack of impact. Discuss shortcomings in key skills like strategic planning, collaboration, or technical acumen. Suggest improvements to better meet the expected standards. Do not simply provide an example of a Bad Answer, but you can provide examples as part of the detail.
  10. Specify the OK/Query Threshold and the Query/Problem Threshold. Both of these numbers should be between 1 and 10. The number provided for the Query/Problem Threshold must be lower than the number provided for the OK/Query Threshold.

For example, if 8 was set for OK/Query Threshold and 3 for Query/Problem Threshold, then the answer will be scored as follows:

  • 8 or above = OK
  • 4-7 = Query
  • 3 and below = Problem
  1. Click Save.

How to Review Entries that the AI has Checked

The AI will assess the answers on entry submission. Depending on the number of submissions and the number of questions on the form that require AI assistance, the length of time that the AI will report its answers will vary. When admins edit an entry, they may see that the AI is still reviewing the answers. If this is the case, then just wait for the AI to update with its answer.

When the AI has reached a decision, the question status will be updated accordingly. There is nothing to prevent Admin Users from overruling the AI and selecting another status.

Below each question, admins will see that one of the AI assessments has been selected to represent the answer. To see all of the assessments, click on Review All Responses.

Admins can request that the AI reassess the answer. This is useful if the AI judging criteria set on the question have been updated. To action this, click on Rereview the answer.