Skip to main content

Industry Benchmark: Which New Zealand web agencies take accessibility seriously?

Joseph Wynn
··4 min read
Industry Benchmark

The Industry Benchmark series takes a sample of websites from an industry, either globally or within a specific region, and evaluates their web accessibility score. The aim of these posts is primarily to gather data on the state of accessibility across the web, but we also hope they can serve as a call to action for the websites that do not score well. This post is the first in the series, and it focuses on web agencies in New Zealand.

Methodology

The agencies that we tested for this benchmark were selected by hand. We tried to select agencies who have done work for large organisations, government agencies, and not-for-profits, since those are the types of clients who are likely to have accessibility requirements. We added each agency into A11y Pulse, and used the site crawl feature to find up to 100 extra pages for each site. A11y Pulse scanned the pages and generated an accessibility score for each one. The overall accessibility score for a website is calculated as the average score of all its pages.

On top of the automated testing, we also used A11y Pulse to guide us through manual testing on the sites that had the best and worst scores. This testing was a combination of keyboard navigation and screen reader use. The screen reader software we used was VoiceOver on Mac in both Firefox and Safari. Manual testing is used purely to provide additional insights and does not affect the accessibility scores.

27
Sites scanned
66
Average accessibility score
56%
Sites with at least one critical issue
26/27
Sites failed the heading orderHeadings convey page structure for screen reader users the same way text size does for sighted users. Properly ordered headings (h1 through h6) allow screen reader users to quickly navigate and understand content structure, saving significant time and frustration while also improving search engine optimization. audit

Accessibility score by site

Of the 27 agencies we tested, only four received a “good” accessibility score of 90 or above. More surprising was the fact that nearly half of the agencies received a “poor” score of less than 70, with the worst-performing agency having over 240 critical accessibility issues on their website.

  1. SilverstripeScore: 96
  2. Somar DigitalScore: 93
  3. Fahrenheit DigitalScore: 92
  4. SquizScore: 92
  5. SignifyScore: 89
  6. Catch DesignScore: 88
  7. CucumberScore: 87
  8. DNAScore: 87
  9. NV InteractiveScore: 87
  10. Catalyst ITScore: 85
  11. SpringloadScore: 84
  12. AssurityScore: 79
  13. VocoScore: 75
  14. RUSHScore: 71
  15. OctaveScore: 71
  16. Terabyte InteractiveScore: 63
  17. WebstruxureScore: 58
  18. FrondeScore: 56
  19. SysdocScore: 54
  20. DesignworksScore: 48
  21. ToastScore: 42
  22. SatelliteScore: 34
  23. Equinox ITScore: 33
  24. Dynamo6Score: 30
  25. ThetaScore: 29
  26. Tangerine DesignScore: 27
  27. DatacomScore: 21

The podium

While no agency scored a perfect 100, the four that share the podium all scored above 90, which is reassuring to see. We performed extra manual testing on these four sites - read on to see our findings.

Silverstripe

silverstripe.com

Screenshot of Silverstripe's home page
Score96Good
Pages scanned66
Issues found27
Pages by score

Silverstripe came out on top with just 27 issues across all of their pages, and no critical issues. During our manual testing we were happy to see that they got the basics right, although we were disappointed that the skip link did not function on most pages. Dialogs on the 'Meet the team' page also did not capture focus correctly, which was frustrating as a keyboard user. Screen reader support was good, with sensible landmarks and labels.

Somar Digital

somar.co.nz

Screenshot of Somar Digital's home page
Score93Good
Pages scanned62
Issues found52
Pages by score

Manual testing showed that Somar Digital put accessibility first with clear focus state, good labelling, and well thought out interactive components. Their accessibility score was only let down by some colour contrast issues on small text, as well as one unlabelled form element which appears to be a genuine frontend bug. Screen reader support was good, although more content landmarks would have made pages easier to navigate.

Fahrenheit Digital

fahrenheit.co.nz

Screenshot of Fahrenheit Digital's home page
Score92Good
Pages scanned42
Issues found41
Pages by score

Fahrenheit Digital's website is a good case study for where automated accessibility testing can miss key issues. A11y Pulse caught some valid issues, like using <a> instead of <button>. It was only during manual testing that we found the site was virtually inaccessible to keyboard users due to a navigation with no tab index. Screen reader support was also poor, with convoluted markup causing simple text to be announced as nested groups and carousel content lacking meaningful labels.

Squiz

squiz.net

Screenshot of Squiz's home page
Score92Good
Pages scanned47
Issues found90
Pages by score

Squiz impressed us with an incredibly well-built site. Our manual testing found no obvious issues, which is reassuring given that Squiz tout their own commitment to accessibility. Their score was only let down by a large number of 'easy fix' issues such as non-hierarchical heading elements. Screen reader support was excellent and all pages were easy to navigate.

Detailed test results

The table below shows the detailed results for each site that we tested, including a list of all issues that were found by A11y Pulse. You can click or tap on any row to expand it.

One interesting insight in this data is how the top 10 sites all achieve a good accessibility score for most of their pages, while almost none of the pages from the remaining 17 sites reached the good score threshold. This sharp divide suggests that the agencies in the top 10 made at least some effort to write accessible code, while the others may not be aware of accessibility requirements at all.

Toggle detailsRankSiteIssues foundScorePages by score
1
Screenshot of Silverstripe's homepage
Silverstripesilverstripe.com
2796Good
2
Screenshot of Somar Digital's homepage
Somar Digitalsomar.co.nz
5293Good
3
Screenshot of Fahrenheit Digital's homepage
Fahrenheit Digitalfahrenheit.co.nz
4192Good
3
Screenshot of Squiz's homepage
Squizsquiz.net
9092Good
5
Screenshot of Signify's homepage
Signifysignify.co.nz
15289Needs improvement
6
Screenshot of Catch Design's homepage
Catch Designcatchdesign.co.nz
8888Needs improvement
7
Screenshot of Cucumber's homepage
Cucumbercucumber.co.nz
5687Needs improvement
7
Screenshot of DNA's homepage
DNAdna.co.nz
11487Needs improvement
7
Screenshot of NV Interactive's homepage
NV Interactivenvinteractive.com
12787Needs improvement
10
Screenshot of Catalyst IT's homepage
Catalyst ITcatalyst.net.nz
21485Needs improvement
11
Screenshot of Springload's homepage
Springloadspringload.co.nz
4784Needs improvement
12
Screenshot of Assurity's homepage
Assurityassurity.nz
4579Needs improvement
13
Screenshot of Voco's homepage
Vocovoco.co.nz
29575Needs improvement
14
Screenshot of RUSH's homepage
RUSHrush.co.nz
22471Needs improvement
14
Screenshot of Octave's homepage
Octaveoctave.nz
24971Needs improvement
16
Screenshot of Terabyte Interactive's homepage
Terabyte Interactiveterabyte.co.nz
26063Poor
17
Screenshot of Webstruxure's homepage
Webstruxurewebstruxure.co.nz
28558Poor
18
Screenshot of Fronde's homepage
Frondefronde.com
30756Poor
19
Screenshot of Sysdoc's homepage
Sysdocsysdoc.com
49054Poor
20
Screenshot of Designworks's homepage
Designworksdesignworks.com
6248Poor
21
Screenshot of Toast's homepage
Toasttoast.co.nz
40342Poor
22
Screenshot of Satellite's homepage
Satellitesatellite.co.nz
57634Poor
23
Screenshot of Equinox IT's homepage
Equinox ITequinox.co.nz
24033Poor
24
Screenshot of Dynamo6's homepage
Dynamo6dynamo6.com
33830Poor
25
Screenshot of Theta's homepage
Thetatheta.co.nz
50329Poor
26
Screenshot of Tangerine Design's homepage
Tangerine Designtangerine.co.nz
29227Poor
27
Screenshot of Datacom's homepage
Datacomdatacom.com
71321Poor

The lowest-scoring sites

Just like we did for the podium websites, we also performed manual testing on the three lowest-scoring sites. We thought the low scores were justified, although we were surprised to find that the worst-performing site was actually more usable than its score implied.

25
Screenshot of Theta

Theta

theta.co.nz

29Poor
Pages scanned75
Total Issues503
Critical Issues36

Severity breakdown

  • Critical36
  • Serious229
  • Moderate238
  • Minor0

During manual testing we noted that Theta's website was mostly accessible for keyboard users, but poor structure and lack of landmarks made screen reader use extremely difficult. Colour contrast was a consistent issue, with important information such as dates and titles being hard to read. While Theta does have a large number of accessibility issues, many of them would be reasonably easy to address and would go a long way to improving their site's usability.

26
Screenshot of Tangerine Design

Tangerine Design

tangerine.co.nz

27Poor
Pages scanned43
Total Issues292
Critical Issues37

Severity breakdown

  • Critical37
  • Serious111
  • Moderate144
  • Minor0

Tangerine Design got a few basics right such as a skip link and keyboard-accessible navigation, however there were some visual design choices that made other parts of their website completely inaccessible to keyboard users. Screen reader support was a mixed bag, with VoiceOver on Firefox reading every element as 'clickable' and VoiceOver on Safari being unable to read any of the content on the page. Tangerine's website needs some significant structural improvements to be accessible to all users.

27
Screenshot of Datacom

Datacom

datacom.com

21Poor
Pages scanned95
Total Issues713
Critical Issues247

Severity breakdown

  • Critical247
  • Serious145
  • Moderate222
  • Minor99

It was not possible to navigate around Datacom's website with a keyboard or screen reader because their navigation uses <div> elements where <button> or <a> should have been used. The content that we were able to test with a screen reader was in poor shape: lack of landmarks made navigation difficult, and no alternative text on images made the reading experience extremely painful. While A11y Pulse did find many valid issues, the single biggest failing of Datacom's website was that nobody tested their navigation with assistive technologies before launch.

The most commonly failed audits

The table below shows the 10 most common issues that A11y Pulse found across all the websites we tested. If you read our manual testing notes, these issues won’t come as any surprise - the majority of issues that we found were related to poor content structure and colour contrast.

  1. 1Heading levels should only increase by oneHeadings convey page structure for screen reader users the same way text size does for sighted users. Properly ordered headings (h1 through h6) allow screen reader users to quickly navigate and understand content structure, saving significant time and frustration while also improving search engine optimization.
    26 / 27
  2. 2Elements must meet minimum color contrast ratio thresholdsPeople with low vision or color blindness cannot read text that lacks sufficient contrast with its background. With nearly three times more people having low vision than total blindness, and 8% of men and 0.4% of women having color deficiencies, adequate contrast is essential for making text readable to millions of users.
    23 / 27
  3. 3Links must have discernible textKeyboard users and screen reader users can only interact with links that have accessible names and can receive focus. Without proper names, users don't know where links lead. Without keyboard focus, users who cannot use a mouse cannot activate the links at all.
    19 / 27
  4. 4All page content should be contained by landmarksContent should be organized within high-level landmark regions like <header>, <navigation>, <main>, and <footer>. Content outside these sections is difficult to find and its purpose may be unclear, making navigation inefficient for screen reader users who rely on landmarks to understand page structure.
    17 / 27
  5. 5Headings should not be emptyScreen readers announce headings to help users navigate page structure. Empty headings confuse users and prevent them from understanding the page's organization. Headings should only be used to convey structure, not for visual styling, and must contain accessible text that screen readers can announce.
    15 / 27
  6. 6Landmarks should have a unique role or role/label/title (i.e. accessible name) combinationEach landmark on a page must have either a unique role or a unique accessible name. Duplicate landmarks confuse screen reader users about which section they're navigating to, making it difficult to efficiently move through the page structure.
    13 / 27
  7. 7Page should contain a level-one headingScreen reader users use keyboard shortcuts to jump directly to the first h1, expecting to land at the main content. Without an h1 or with a misplaced one, users must listen to more of the page to understand its structure, wasting valuable time on every page visit.
    12 / 27
  8. 8Frames must have an accessible nameScreen reader users rely on frame titles to understand frame content without exploring each one. Without descriptive, unique titles, users receive unhelpful information like "frame," "JavaScript," or URLs, making navigation through frames difficult and confusing.
    11 / 27
  9. 9Document should have one main landmarkEvery page should have exactly one main landmark that identifies the primary content. Without a main landmark, screen reader users must navigate through all content linearly to find what they're looking for, making page navigation difficult and time-consuming.
    10 / 27
  10. 10Images must have alternative textScreen readers cannot translate images into text without alt attributes. Alternative text is essential for blind users and beneficial for users with low vision or color blindness, providing the same information that sighted users get from viewing the image.
    8 / 27

Our conclusion

Based on the test results, it feels like accessibility on the web is still very much split between those who make an effort and those who don’t. During manual testing we could usually tell within the first 30 seconds whether a site had been built with assistive technologies in mind, and most of the time this was reflected in the accessibility score.