Sometimes a client comes to us with a technical vendor who has an online judging tool they plan to use. The problem with that scenario is that their vendor typically thinks like a technical expert and not like a contest expert. We believe that judging should be approached from a marketing perspective as well as a technical perspective! And the online judging tool should be built accordingly — like the Marden-Kane judging tool is! Contact us for a demo and details so you won’t have to worry about any of the issues listed below that we have seen caused by the other guys.

As part of our judging services for clients we have actually used some other judging tools built by technical companies in order to judge entries and found many of them are lacking in the following areas:

1. Ability for multiple judges to score

This is complaint #1 for third-party tools we have used for our client contests. We ask for log in credentials and are told we will be sharing a log in user name and password. How is that supposed to work? How will we know who judged what? And what will happen if we all try to log in at once? In one word what happens is chaos.

A judging tool should have the ability for two or more judges to be able to log in at the same time using unique user names to judge entries. Part of this is just common sense, in that you can’t expect judges to take turns judging – usually you have a lot of entries to judge in a short amount of time and limiting the tool to only one user doesn’t make sense. The system should also be able to tell which judge scores what in case there are ever questions about why an entry was scored a certain way.

2. Entry locking

Along the same lines as having the ability for multiple judges to score simultaneously, there should be a way for the tool to “lock” each individual entry as it is being scored. By “locking” I mean two judges can’t be looking at the same entry at the same time; if Judge A is looking at Entry #1, Judge B should be directed to Entry #2. That way, if each entry only needs one score, you don’t have multiple judges all seeing and overwriting each other’s scores or causing errors. Plus, if you are in a time crunch it is a huge waste of everyone’s time to be looking at the same entry when you all could be scoring multiple entries at the same time!

3. Ability for entries to be scored more than once

There are times when you require that each of your celebrity judges needs to score every single entry, or you might need to have multiple rounds of judging, but some online tools are only set up for entries to be scored by one judge per entry with only one round of judging. Make sure there is a setting in the tool that can be changed so that you can have each entry judged by each judge, or by at least 2 or more judges, or only one judge – in any case, make sure there is some flexibility.

4. Custom scoring capabilities

Each contest is different, so being allowed only a set number of judging criteria or a scale of 1-10 only for each criteria may not work for your contest. The scoring needs to be set up in the tool exactly as it is in the rules. If you have two criteria and Criteria #1 is supposed to be scored from 1-10 but equates to 60% of the score, Criteria #2 is scored 1-20 but equates to 40% of the overall score, then make sure the tool reflects that. I have seen many tools where the scale is preset and can’t be changed no matter what the rules say.

5. Ranked Entries

By the way, a list of ranked entries based on judges scores should be a given after judging is completed. An online judging tool should be able to take the scores and calculate the top entries for you. If you have to download the entries with raw scores and then do the math in excel, then the tool is not doing its job, and neither is that vendor!

6. Ability to segment entries by category or language

Nothing is more frustrating than getting a few expert judges lined up to score the entries that came in to review French and Spanish or by area of expertise, only to realize that there is no way to categorize the entries in the online judging tool and those judges have to hunt through all of the entries to find the few in their language or topic. A good tool should have a way to partition out entries by categories like language or topic or geographic area. You want your specialty judges to only have to view the entries pertinent to them.

7. Tracking and Reporting Data

Knowing how many entries each judge scores, and the range of their scores can be extremely useful. If that data is not available to someone, then things that could have been caught early might be missed.  For example, while using the Marden-Kane judging tool where we have reporting data available, we had a judge not using the full range of possible scores because she hadn’t noticed the scrollbar on the page where the scoring criteria were listed. Once we saw there may be an issue and addressed it, it was fixed after only a few entries had been scored by her, rather than after she completed all of her judging.

8. Search Functionality

Sometimes at the end of judging your celebrity judge will say, I really wasn’t sure about that entry I saw about XXX. Whatever XXX was, it is usually something REALLY bad that the entrant was saying about the brand. You will want to find it to bring it to the client’s attention, but without a search this would be impossible if you have thousands of entries and the judge has no idea which one it was. Save yourself some time and ask for the ability to search through entries efficiently.

9. Escalation of problem entries

And speaking of problem entries, there are times when you get them and they need to be brought to the attention of a client FAST. None of the third party tools we have seen ever had a way to do that quickly. A good judging tool should have a way to flag entries or mark them as suspicious. And if it’s truly custom, a way to have a login for the client or their legal team to go in and view those suspicious entries would be ideal.

10. Security

If your judging tool is not secure, it could be hacked, making your simple contest into a nightmare when personally identifying information such as name and photos and essays and scores fall into the wrong hands. Insist on security, including a security certificate on the judging tool web site, unique user names and passwords for each judge, and use of secure password resets.

Bonus Reason: Browser compatibility issues

I am still not sure how this happens, but some of these technical vendors still develop tools that can only be used in one type of browser. This makes me wonder how they call themselves a technical vendor because browser compatibility is a technical thing, but I digress. The tool should work cross browser in all current major browsers (IE, Chrome, Firefox and Safari). Period.

 

The best way to solve all of these problems is to use the contest judging tool designed and built by the contest experts at Marden-Kane. The Marden-Kane judging tool is customizable, cost effective and it addresses all of these concerns (including automatic escalation of entries if needed) so you won’t be wasting your precious time and money.

Contact us for a demo to see how our tool works better than the competition!

To read more posts by Marden-Kane, please visit our main blog page or subscribe to our email list.

Save