How to Manage Scoring and Comments for Judges

Admins have a variety of features in the Viewing Interface that Judges can use to provide feedback on Entries. One of the most popular is Scoring, where Judges provide scores on criteria set by Admin Users. Admins can fully customise the number of scores per Entry, and the range of each score, allowing detailed feedback from Judges. In addition, Admins can request comments from Judges to use as quotes in the brochure or to have a greater understanding of their decisions. Scores and comments can be set independently of one another, so there is no need to set both.

Once scores have been supplied, reports can be run to extract the results allowing Admins to identify winners, runners up and nominees. Scores can be set to be mandatory by requiring Judges to submit. Alternatively, they can be set as optional, where Judges can chose if they want to provide a comment or a score.

How to Enable Submission of Scoring and Comments

Selecting this option will mean that Judges are required to submit their scores and comments. This means they will see messaging that informs them that their scores are not complete until they have been submitted.
Note there is no functionality to set a deadline, so Admins must chase users to submit.
Automated emails can be sent via Nucleus, and there are tools for Admins to see the status of each Judges scoring progress. In addition, there are features which assist Judges in managing their scores and comments, as this can become complex if Judges are reviewing many Entries in multiple Categories – these are described in the below sections. It would be a good practise for Admins to consider beforehand how many Entries each Judge is required to review.

If you do not want your Judges to submit the scores and comments then you can skip this section.

Log into the Admin Interface
Go to Actions > Awards.
From the Awards List, find the Award where you want to enable Scoring and Commenting
From the Actions column on the right, click Edit
Scroll down to the Viewing section
Select Yes for Require scores & comments to be submitted
You will need to provide mandatory copy for the following instructions that will be displayed to Jurors at key stages during their scoring process (150 characters max):
- Not started scoring message – displayed when the Judge has not started scoring
- Please finish scoring message – displayed when the Judge has scored one or more entries but has not completed the process
- Please submit scores message – displayed when the Judge has scored all entries and needs to submit
- Thanks for scoring message – displayed when the Judge has submitted their scores
Click Save and Close

How to Set Up Scoring and Comments for a Category

Scoring is set up on each individual Category. This allows Admins to set up different scoring criteria on a per Category basis, or to opt not to allow Scoring on particular Categories. Scoring can be set up on a Category regardless whether the Award is set to mandate that Judges submit their scores or not.
The first time Admins will need to set up Scoring on each Category. This setup will be then copied when the Award is copied in the following year. Note that if an Award is to submit scores, then Judges will need to submit all of their scores for all Categories in one action. To set up scores:

Log into the Admin Interface
Go to Actions>Categories
Click on the Edit button in the right hand column for the Category where you wish to set the scores
Scroll down to the Scoring With Viewing section
Edit the Section title of the Scoring and Comments section as you want it to appear to your Judges on the Viewing Interface
Decide if you want to include Scoring. If you select Yes, some new fields will open enabling you to:
- Name the New Score Title – which may be a criteria, such as ‘Creativity’, ‘Originality’, ‘Budgetability’ etc
- Select the Min Score – i.e. the minimum score an Entry may be given on a scale
- Select the Max Score – i.e. the maximum score an Entry may be given on a scale
To add another, simply click Save and another score will be available to create
To enable comments set Commenting to Yes
Provide a Comments title for this section or leave the default
Set the Max Comment Length in characters
When you have created your Scoring and Commenting settings, click Save & Close

How to Enable Judges to Flag a Conflict of Interest

There will be occasions where Judges have an attachment to an Entry which constitutes a conflict of interest. This could be because the Judge has worked on the Entry, or works for the same company that produced the Entry, or is related to one of the Entrants. Depending on your Awards workflow, there will be different mechanisms to deal with this. Nucleus provides Judges with a simple mechanism that allows them to alert Admins of any conflicts.

Switching on this feature will display a checkbox next to each Entry, which Judges can use to confirm their conflict. Once confirmed, the Entry is highlighted in red for Viewers and the scores and comments are locked. The scores for an Entry that is marked as a conflict of interest will not be included in the results file. The file will also highlight where a Judge has marked an Entry as a conflict. To enable this feature:

Log into the Admin Interface
Go to Actions>Awards
In the table click Edit on the relevant Award
In the Viewing section set Allow Jurors to Flag Conflict of Interest to Yes
Click Save

Note – at least one category in the Award must have Scores or Comments enabled in order for this feature to appear on the Awards Setting page. Once enabled, the conflict of interest will appear for all Categories in an Award.

How to Set Up Functionality that Assists Judges with their Scoring and Comments

Nucleus provides a range of functionality in the UI that assists Judges with managing their scores and comments. If a Judge is reviewing one Category with 10 Entries, then it is easy to track progress.
If a Judge has multiple Categories and many Entries, it can be easy to lose progress, especially between sessions. Therefore, Nucleus provides default and optional functionality to assist Judges:

Entry Confirmation Tick – a tick will appear in the scoring section for each Entry when Judges have completed all score dropdowns and comments
Category Progress Bar – a bar and a percentage is shown on each Category block so that Judges can easily see their progress in any Category
Scoring Summary Screen – A screen which displays all Entries in each Category a Judge is scoring which details the progress and highlights Entries that are missing scores and comments
Save Button for Each Score – Allows Judges to save and update their progress regularly

Optional Functionality:

Viewing Site Order – allows Judges to filter Entries by scored/not scored and a range of other options
Alphabetical Navigation – allows Judges to to filter for Entries alphabetically
Tagging – create tags for Judges to filter Entries

For more information on the above features and many more Viewing Interface features see the Viewing Interface guide.

How to Review Judges Scoring Progress

If needed, during the Judging process you can check on how far your Viewers have progressed through the scoring/commenting of their allocated Categories. To see the progress of your Viewers’ activity during the scoring/commenting stage, do as follows:

Log into the Admin Interface
Go to Actions>Viewers
The Viewers List displays. Select the Award you want to check by filtering it from the Award(s) column.
If required, do likewise with the Category dropdown menu
The ‘% complete’ column displays. Here you can find your Viewers’ progress expressed as a percentage

You can also filter the list to display only those Viewers by scoring percentage.
a) To select those who have viewed less than a certain percentage:

Select the < under the % complete filter
Type in the percentage figure in the box alongside

b) To select those who have viewed more than a certain percentage:

Select the > under the % complete filter
Type in the percentage figure in the box alongside

How to Contact Judges based on their Scoring Progress

Admins can contact Judges based on their scoring progress to remind them to submit scores or chase based on an upcoming Submission Deadline. It is possible to set up recurring emails triggered at a pre-set time and recurrence or to create one off email jobs. For more information about using the Entrant Mailout functionality, read this guide.

How to Unsubmit Scores for Judges to Edit

Once a Judge has submitted their scores and comments, they become read-only. Judges can still see their scores and comments, but they are unable to edit them. If a Judge contacts Admins to request that they update their scores, Nucleus facilitates this by allowing Judges to edit and resubmit their scores so long as Admin Users approve this action.

Log into the Admin Interface
Go to Actions>Viewers
Filter for the relevant user and click Edit in the actions menu
In the right hand column search for the relevant Award
Click on the Unsubmit Scores and Comments button
The Judge will be able to edit the scores again

Note – the unsubmission process will trigger the viewer/unsubmitScoresComments email template. Please review and update or disable the template before unsubmitting the scores. More information about editing email templates can be found in this guide.

How to Run Reports for Judges’ Scores

There are two reports available for Admins to be able to extract score data:

Full Score Details – exports all score data submitted by judges
Summary (Average scores) – combines the Judges’ scores for each Entry and outputs an average score

If you require a bespoke report to assist in your workflows, please raise a support ticket as this can be commissioned.

To export a Report:

Log into the Admin Interface
Go to Actions>Misc
Click on Viewer Score Download in the Reports section
Select the relevant Award
Select the Categories to export
Choose the Report format
Click Download