Copyright American Society of Association Executives Jun 2002| [Headnote] |
| ANSWERING KEY QUESTIONS CAN LEAD TO WEB SITE EFFECTIVENESS. |
Association Web sites have become necessity rather than luxury. And while many association executives have adjusted to the fact that a Web site is a big-ticket item in terms of monetary investment, some are still struggling with the concept that it is also a big impact item in that it reaches all aspects of the organization. No longer is Web site development considered a project specifically for the information technology or the communication department. Rather it requires involvement from the CEO to the data entry person to the association's various constituencies. The association's Web site is far-reaching and should be regarded as the virtual embodiment of the organization, from strategy to process.
Consequently, many association leaders ask themselves just how their Web sites stack up when it comes to providing member benefits and services, facilitating the right kinds of communication, and supplying information both to members and the greater public. This is no longer a question answered simply by counting the number of hits to a particular Web page. Rather, the effectiveness of a Web site must be evaluated with multidimensional measures.
In response to several inquiries from ASAE's membership about the effectiveness of their Web sites, the ASAE 2001-2002 Technology Section Council created a tool for association leaders to use in evaluating their sites. This document, organized in seven categories, attempts to identify many key questions-- ranging from specific technical questions to those centered on business and strategy-that an association should ask about its Web site. As with many other functions of an association, the one-size-fits-all approach doesn't work. One association Web site may focus on sales while another focuses on the dissemination of information. Consequently, these sets of questions should be used as a self-guided tool to establish your associations' unique benchmarks. One of the main edicts relating to the evaluation tool is that the Web site's effectiveness should be considered from the executive's bird's-eye view, as well as from the nitty-gritty perspective of the day-to-day, hands-on manager. Additionally, the site should be reviewed periodically and results should be compared to earlier results. Exactly how this is done will vary from organization to organization. Perhaps the Web manager will poll the executive director regularly regarding the business-related issues, or all functional area representatives will meet on a periodic basis to expand the set of questions that should be asked based on unique experiences, and so on. And just as the organization's overall strategic plan should be a living document, so should this evaluation be. The bottom line: By using this evaluation tool, your association should benchmark those qualities that it would like to improve and set goals and strategies for improvement.
Here we outline four of the seven sections for evaluation and provide a couple of case studies of how the tool might be applied to a specific association's Web site. For the complete document, go to [www.asaenet.org/sections/technology/resources.]
Business planning
When asked the question, "Does your organization have a strategic plan?" followed quickly by "... and have you taken it into consideration when developing a plan for your Web site?" the response frequently takes the form of raised eyebrows, followed quickly by a furrowed brow. In an environment sometimes slow to evolve from its silo culture, those in charge of managing the organization's Web site may or may not be those in charge of technology-who, in turn, are not likely to be those in charge of the organization's strategic plan. Moreover, creating synergy between the organization's overall plan and that of the Web site has typically not been viewed as necessary. It is, however, difficult to debate that technology provides an infrastructure for the organization's daily operations and is often the instrument for playing out future strategic goals. Not only is the Web site the organization's face to the outside world, it is a vehicle for supporting future strategy. Given this, it is critical that an organization align the strategic plans of both its Web site and its organization. Asking the following questions, responding to the answers, and returning to these questions on a regular basis are critical to the ongoing assessment of the effectiveness of the organization's Web site.
1. Do you have a strategic plan for your organization?
2. Do you have a strategic plan for technology?
3. Do you have a strategic plan for your Web site?
4. Does your Web site strategic plan support your overall strategic plan?
5. Have you defined your Web site audience(s)?
6. Have you defined your Web site objective(s)?
7. Are your objectives) aligned with the expectations of your audience(s)?
8.Which functional areas are involved in defining your objectives (e.g., marketing, membership, public policy)?
9. Was your board involved in defining your objective(s)?
10. Have you evaluated competitive Web sites?
11. What value can your organization deliver to your members or customers through your Web site?
12. Are there services that you offer only through your Web site?
13. Do you have a budget commitment to achieve your objectives and reach your audiences?
For an example of how this questioning process might be incorporated into an effective Web site redesign, see sidebar, "Aligned Strategies Support Association's Goals."
Usability
A Web site is of little value to an association unless it gets used. There are several ways to gauge how your site is used. Success cannot be judged by the number of Web hits alone, as Web hits indicate only the number of files downloaded from your site. A page with several graphics will generate several hits, although the page was only viewed once by the user. Search engine crawlers (systems that look at each page on your Web site to create a searchable index) can generate several thousand hits and skew your analysis. Better measures include the number of unique visitors and the number of page views. Those measures can give you a better idea of how many people you are reaching with your site.
Navigation plays an important part in usability and can make or break your Web site. Navigation is most useful when it is transparent to the user and provides the maximum flexibility in viewing the site. Remember that people's learning styles differ; some prefer graphical navigation while others prefer textual navigation aids. Make sure that your Web site doesn't turn off any group from using your site.
Everything on the site should be easy to find. Web sites accomplish this through well-designed, intuitive interfaces that allow the user to get to any of the content in just a few clicks. A site search option will allow users to quickly get to the content that they know exists on the site. A detailed site map can be a valuable tool for users who want to browse through the site and learn about it. Consider conducting usability tests of your current and proposed new sites with your members. Their input can help you understand which navigation queues work and which don't work, allowing you to make useful changes to your site.
Ask yourself the following questions regarding the usability of your association's Web site:
1. Do you analyze regularly or do you have standards for the amount of downtime users experience, bandwidth usage, page-- load time, number of unique visitors, number of page views or sessions, time spent on the site, graphics optimization, repeat visitors, origin of visitors (domestic versus international), and the most-accessed and least-accessed pages?
2. Have you set a standard as to how quickly you respond to visitor queries?
3. Do you have a process in place for reviewing and correcting Web site errors?
4. Has your number of completed transactions versus incomplete transactions improved across time?
5. Is your site easy to navigate?
6. Are all your links labeled clearly?
7. Is valuable information located three clicks or less away from the home page?
8. Does your site include an index or site map?
9. Does your site meet the WC3/WAI (Web accessibility) guidelines?
10. Have you checked to make sure your Web site works on different browsers, platforms, and monitors?
11. Have you conducted a needs assessment and usability assessment of your target audience?
| [Photograph] |
| CONSIDER CONDUCTING USABILITY TESTS OF YOUR CURRENT AND PROPOSED NEW SITES WITH YOUR MEMBERS. THEIR INPUT CAN HELP YOU UNDERSTAND WHICH NAVIGATION QUEUES WORK AND WHICH DON'T WORK, ALLOWING YOU TO MAKE USEFUL CHANGES TO YOUR SITE. |
| [Photograph] |
| A WELL-DESIGNED SITE WITH LITTLE FUNCTIONALITY IS OF LESS USE THAN A SITE WITH A LOT OF FUNCTION AND LITTLE DESIGN. |
12. Have you measured your site against the Fogg index? (This is an index that relates to the readability of the site.)
13. If appropriate for your audience, is your site available in multiple languages?
14. Is your site optimized for search engines via appropriate page titles, meta tags, meta definitions, and keywords?
15. Does your home page clearly identify the name of your association and what your association does?
16. Does your site have easily accessible (within one click) adequate contact information, including your association's address, phone, fax, e-mail, and staff contact names?
17. Do you provide a way for visitors to give feedback?
18. Is your Web site a collaborative effort between departments?
Visitor experience
The probability that a visitor will have a positive experience on your Web site will be based on a number of factors. For example, the visitor's technological background and expertise will have much to do with how the experience goes. And the best way to find out what visitors think is to ask them. Consider implementing a feedback mechanism on your Web site to let people tell you what they like and don't like about your site. An annual survey of your constituents and targeted focus groups can help you identify what is working and what is not working on the site.
The style of your Web site should be closely tied to the site's goals. A Web site, for example, that primarily focuses on news about an industry will look different from one that provides professional development for members. Since the functionality of your site should be more important than the design, more Web designers are using the form-follows-function rule. A welldesigned site with little functionality is of less use than a site with a lot of function and little design. A well-defined style sheet that includes color, font, graphical, and navigation schemes is a must for association Web sites.
The visitor experience is usually enhanced by providing fresh content that is valued by members. A home page that changes often (weekly or daily) can drive to our site repeat visitors who know that the information will be updated. In general, it's best if the content is published closest to the origination source. In other words, if the meeting planner recognizes that the hotel phone number for the conference needs to be changed, it would be best if he or she could make the change rather than having to send the correction to another part of the organization for someone else to post. That, however, assumes a well-designed business process and content flow through the organization, including writing, editing, and review. Many associations are accomplishing this through a content management system that automates these functions.
Associations are demonstrating value through other techniques as well. Some provide a members-only area, allowing only authenticated users to visit the section that gives access to their Web-delivered membership benefits. Associations are beginning to tie this authentication to their association management system, a practice that requires tight integration of the Web site with business processes of the membership department. Many Web sites are adding community-building tools that facilitate online discussions and collaborative workspaces where members can discuss common problems and solutions or work together on mutual projects.
Questions to start a conversation about evaluating your Web site visitors experience fall in several categories.
Understanding your members and visitors
* Have you surveyed your constituents?
* Do you know why visitors go to your site?
* Do you know why nonvisitors do not?
Achieving visual appeal
* Does your site reflect your association brand?
* Is the color scheme effective for viewing?
* Have you measured visitors perceptions?
* Do you have a site style guide?
Providing valuable content
* Do you have a member-only section and why?
* What information is in the member section and why?
* What information is available to the public and why?
* Do you have community-building features on your Web site (e.g., listserver, discussion forum, polling)?
* Can you search your content meaningfully?
* Do you have processes for quality control including proofreading?
* Is there a person that has a strategic overview of the Web site?
* Do you have an automated content management system?
Updating information appropriately
* What is the frequency of update of your home page and other pages?
* What percent of your Web site is created dynamically?
* What percent of the staff can update their own content on the site?
* What percent of the staff actually update their own content?
* Have you set standards for how long it takes to get something posted?
* Are your updaters adequately trained?
* Do you have a Web site maintenance schedule that details when updates should be made to content? (This can include technical maintenance aspects as well.)
To see how the answers to some of these questions can guide Web site revision, read the sidebar, "Welcoming and Listening to Your Web Site Visitors."
Synchronization of the Web site with strategic goals
While business planning speaks to the organization's initial strategizing, this section questions specific outcomes related to the overall plan. Perhaps more than in other sections, the questions are suggestions and should be enhanced or replaced, depending on the goals outlined in the organization's strategic plan. For example, an organization primarily involved in fundraising that may not even have members would need to substitute relevant questions in the "member recruitment" section below. Perhaps question one, for example, could be modified to read: "Are you recruiting new donors via your Web site?"
The message should be clear now: Those in charge of developing, maintaining, and monitoring their organization's Web site should become quite familiar with the organization's strategic plan and should elicit the input and participation of other staff members-from those responsible for handling the orders being entered via the Web today to the executive director who has a vision for where the organization needs to be tomorrow. Ask your Web team the questions in the following categories:
Member recruitment
* Are you recruiting new members via your Web site?
* What percentage of your new members joined as a result of your Web site?
* Have you opened new membership markets because of your Web site?
* Does your Web site provide direct member services?
* Does your Web site facilitate chapter recruitment?
New business opportunities
* Have you opened new business markets because of your Web site?
* What percentage of your association's sponsorship opportunities are related to your Web site?
* Have you changed any fee-based services to free Web-based member benefits because of your Web site?
Miscellaneous
* Is the URL on all publications?
* What unexpected benefits accrued from the design or implementation of your Web site?
To sum it up, your association Web site is a work in progress. Evaluating it on an ongoing basis can be facilitated by building conversations and decisions around the key questions included in this comprehensive Web site evaluation tool.
| [Sidebar] |
| Aligned Strategies Support Association's Goals |
| [Sidebar] |
| For the American Speech-Language-Hearing Association, Rockville, Maryland, Web site goals were a natural extension of the organization's strategic goals. "ASHA's executive board decided a few years ago," says David Gammel, Web site director, "that they really wanted to drive ASHA toward using the Web as its primary vehicle for communication with its 103,000 members and with the general public-as well as for delivering services to members." As part of that strategic initiative, ASHA allocated significant resources for a major redesign of ASHA's Web site beginning in 2000-and launched the redesigned and enhanced site (www.asha.org) in mid-200.. Gammel and his team put in place resources, processes, and evaluation tools that they continue to revise as they learn more along the way. |
| Staffing up |
| [Sidebar] |
| Putting resources where its priorities were, ASHA began staffing up in spring of 2000. "The Web team," says Gammel, "was two and one half full-time-equivalent employees: a director, a content editor, and a graphic designer. Now the team has become a cluster, our term for a department, and includes six people. This staffing up took place in conjunction with our work on Web site redesign." |
| The board's enthusiasm for the project motivated the formation of an internal Web Strategy Team, including ASHA's execu |
| [Sidebar] |
| five director, several staff officers, and several director-level staff. "This group of 12 or 13 people," says Gammel, "translated ASHA's priorities into a strategic plan for the Web, outlining specific goals. The mission that summed it all up became: The ASHA Web site will be viewed as one of the most valuable member benefits that members receive from the organization." |
| Determining goals |
| [Sidebar] |
| From the beginning, says Gammel, the Web team gathered feedback from throughout the organization. "We talked with directors and chief staff officers, gathering their feedback on the site and the priorities for their units. We talked with line staff from various units about their interests in the Web site and their interactions with members with regard to the site." Gammel's group also reviewed survey research that ASHA had done with its members, focused on priorities members had identified-and also took a look at individual member feedback through anecdotal comments received via the Web site and through other interactions with staff. "I wouldn't call it a highly organized process," says Gammel, "but we gathered feedback from what we thought were many important areas." |
| The site that ASHA was replacing "had grown in an organic manner," says Gammel. "This is a pitfall for many associations in that sites often end up looking like an organizational chart |
| [Sidebar] |
| because that's where the content comes from." ASHA's goals included reorganizing content and making it easier for members to access it using their own logic rather than the logic that is driven by the structure of the staff. "In addition," says Gammel, "we needed to respond to member feedback that indicated people were having a hard time getting from our home page to the professional site that was targeted to members." While the previous home page was targeted to ASHA's consumers-people who are personally impacted by some kind of communication disorder-the new site has a splash page that divides Web users into three different audiences, giving each a route to their particular section of the site. "Now it's very clear," says Gammel, "that if you are a professional, you click on the designated section." |
| Fulfilling the strategic goals |
| [Sidebar] |
| At this point, the ASHA site has gained some critical mass when it comes to delivering member services. "A major benefit," explains Gammel, "is that our scholarly journals are now available in an online archive. Members can log in and access back issues of our journals and individual articles. Our academic members really love that. It makes conducting research much easier for them-and other members who are practicing in the field can access articles and information to refresh their knowledge on given topics." |
| ASHA's previous site had already provided an online membership directory and electronic processing of submissions of academic papers. Now, with the site's clear entry points, |
| [Sidebar] |
| enhanced navigation, and additional information, Gammel thinks the site has come along way toward reaching its original goals-as well as ASHA's strategic goal for the site. |
| Looking forward |
| [Sidebar] |
| Despite the site's progress, Gammel says, "We're definitely not completely there [in supporting the strategic plan]. We haven't gotten our survey back of what member perceptions are about where we are. From my own perspective, there's a lot more work we can do, and it's going to be a long-term process-- including analyzing formal feedback from member surveys, conducting member focus groups, and doing staff interviews." |
| Gammel is particularly excited about the process ASHA has established for conducting the next redesign cycle. "We've hired a consultant to help us craft a review process for analyzing all the feedback that we collect," he says. "She will review the feedback based on established principles of Web usability and then make recommendations on how to modify our site based on these findings." This process includes creating a prototype of the new site and then having members come in and actually try out the site. "Basically," says Gammel, "we'll give them certain tasks to achieve, and if they run into difficulty we'll know that we need to fine tune the design before launching it to the entire membership. This annual process," says Gammel, "will incorporate user comments and requirements into a formal review, so that the ASHA Web site will evolve according to a well thought-out plan rather than by a more random process of adding things over time." |
| [Sidebar] |
| Welcoming and Listening to Your Web Site Visitors |
| [Sidebar] |
| About 18 months ago, The Association for Work Process Improvement (TAWPI), Boston, completed a major renovation of the organization's Web site (www.tawpi.org). "The completely new design," says Linda N. O'Hara, president, "made it much easier to navigate, added several new sections-specifically a members-only section-and improved our shopping cart funtion." While O'Hara plans now to seek member feedback on the new site, the current redesign was staff-driven. Here O'Hara explains why the decision process-driving not only the visitor experience, but many other aspects of the Web-may change depending on the lifecycle of your Web site. She also talks about tools that monitor the visitor experience and what those results might suggest in terms of modifications. |
| [Sidebar] |
| Responding to complexities |
| When TAWPI began contemplating its recent redesign, "we actually did not survey members," says O'Hara. "We felt that there were certain decisions that the staff needed to make. While our mission is to enhance the performance of organizations and strengthen the values of professionals that employ emerging technologies in mail, remittance, document, and forms processing-our association fulfills that mission for multiple industries (government, insurance, health care, banking, and so on). Any given member wouldn't necessarily have the perspective to make certain decisions about our site." One of the things the staff of seven struggled with was how to set up the site. "Should we set it up by vertical markets that we serve, or by technology solutions, or by other ways?" O'Hara recalls asking. "It was a tough decision. We eventually decided to set up the site based on the categories of our products and services, |
| [Sidebar] |
| because they go across those industry lines. While we are doing a bit more of the vertical organization on the site now, we primarily decided among the staff that we would design the site the way we did." |
| Now that the new site has been up for awhile, TAWPI is seeking member input as to how people use the Web and what they think of the site. |
| [Sidebar] |
| Enhancing the visitor experience |
| TAWPI's site has traditionally provided online registration for events, purchasing of its publications, and a preshow exhibition floor plan allowing members and exhibitors to plot out their meeting itineraries. "With the revised site, we've enhanced our online member update capability," explains O'Hara. "We upload the member database periodically so that members can pull their own records and update their information." In addition, news is changed much more frequently than in the past. |
| "We don't look only at what is on the Web," says O'Hara, "but we look at the peripheral aspects to see how we can drive more traffic to the Web-and how we can save money at the same time." For example, when TAWPI sends out its e-newsletters to members and customers, links provided in the newsletter direct people to certain sections of the Web site. "We also put items in the newsletter," says O'Hara, "that represent what other members are doing on the Web. We have, for instance, a `favorite downloads' section that links the member to the Web and tells him or her what archived articles from the member magazine have been downloaded most frequently during the past month. This, of course, feeds on itself, encouraging more downloads, so it all has a snowball effect." |
| Modifying and improving |
| To continue adding member value and |
| [Sidebar] |
| enhancing visitor experience, TAWPI installed a software package that monitors Web activity. "We run the reports ourselves," says O'Hara, "and they measure absolutely everything you can possibly think of." The program tracks hits on the entire site, average hits per day, average length of visits, international visits, and much more. And what's the value in all of that? O'Hara says that she and her staff have made certain modifications as a result of reviewing the data. "We have modified the time during the week that we send out our e-newsletters," says O'Hara, "based on activity recorded by the day of the week that indicates the time most people are likely to link to our site." O'Hara admits that it's not always straightforward information you can easily act on. "In some ways we drive activity by things we send out from here, such as marketing materials," she says. "So it's difficult to know whether activity is there because people have gotten something from us-or because that's the most common time that they visit the site." |
| Regardless of the challenges, TAWPI will continue to survey its 1,100 individual members and mo affiliate members-both through a yearly survey conducted at its annual forum and exposition and an online survey provided to people who aren't able to attend the show. |
| "My biggest priority now," says O'Hara, "is to spend a year without spending major money on the Web. We've put a lot of resources into the site and feel that we've reached a good level of service. I'd like any additional enhancements to include benefit for the association. For example, if we have a section on the site that allows visitors to send us questions by e-mail, we may require that they provide us with demographic information prior to being able to ask the question." |
| [Author Affiliation] |
| Loretta M. DeLuca is president, DelCor Technology Solutions, Silver Spring, Maryland. E-mail: ldeluca@delcortech.com. Randy Richter is director of information systems and Internet strategies, EDUCAUSE, Boulder, Colorado. E-mail: rrichter@educause.edu. DeLuca and Richter currently serve on the 2001-2002 ASAE Technology Section Council. |
My Research