| A resource kit for evaluating community IT projects |
|
 |
Examples using the EvaluateIT resource kit Here are examples of how community groups can use EvaluateIT to help them review an IT project in their community.
[Note: These are all fictional examples, even where they appear to resemble actual situations.]
by Jill Korgan, Secretary, Westana Shire Development Association Background This is an account of how people in our Westana community worked together to find ways of improving our community IT (information technology) resources and skills. We've been doing this through reviewing and evaluating some of our main IT projects. This story explains how we went about this from when we first thought about doing the review to when we completed it. The Westana Shire has been badly hit in recent years by drought and bushfires. However, we were lucky in having some enthusiastic people who were determined to make sure the district held together and grew rather than declined. They included Councillors and staff in the Westana Shire Council, our local library staff, school principals and teachers, and members of our many community organisations and sporting clubs. For the area to flourish, our group believed that people in the different areas of the Shire needed to keep in touch more and work together better to make the most of the resources in our community. We were interested in the possible benefits of using the Internet for communicating and sharing information, and also in training courses on how to use new computer technologies. Two other examples of community website projects that we found useful were:
Note: Westana is a fictional community but parts of this story have been adapted from actual examples of rural communities reviewing their IT projects.
Our story begins when Helen Park, one of our Councillors in the Westana Shire Council, thought that the IT projects which had begun with such enthusiasm might come to a halt unless something was done to sustain them. Councillor Park thought that a review of the current projects was needed and wanted to get the community involved. David Cordell, the Council's Community Development Officer got together a small group of other interested people, including me, to come up with some ideas about how we could generate interest in the review project. We suggested holding information sessions and a workshop to see how much interest there was in the community. We thought it would be more successful in the long run if we could get a mixed group of people from nearly every sector of the community to join us. After the information session and the workshops, we found we had a group of committed people who were keen to review the IT projects we were involved in. We had mixed backgrounds, experiences and skills. A few were familiar with IT and some of us knew very little, but saw its potential for our families, our businesses and our community. Some were from Westana, the main town in the Shire, and some were from smaller towns in the Shire, or from rural properties.
At first it wasn't easy to decide what we would review. The Westana Shire website had grown tremendously in so many directions that it was now called a community portal. It had many home pages of dozens of different community groups and included online discussion groups for some interest groups. Several different training courses in IT had been held. They included an Internet training course at the library run by Carol McLean, the Council IT Support Officer, and a computer basics course held at the Westana State School by Alan Thompson, a teacher at the school. The library course was funded by the Council and the School course was held with State government funding obtained by the Westana Shire Development Association. The situation had become very complicated with dozens of people being affected. We were concerned that some of those affected would not be interested in being involved in the review. In the end, we decided to concentrate on the IT projects which seemed most manageable, given the people who were interested in the review project. They were:
There were different reasons why each of these projects needed to be reviewed.
Before we undertook the IT projects review, we needed to think about any risks that could be involved and who could help us identify and manage them. Some of the risks we considered were:
At our first meeting David, the Community Development Officer, helped us think through the possible risks and how we could access the resources we needed. We later consulted Dr Sally Green, a researcher from a university in our region, when we were planning the survey and interviews as part of our review.
Once we were clear about the what and why of the review, the next step was to have people from all possible sectors of the community involved in the review, especially if they could benefit from the IT projects or from using IT. From the beginning, we had kept on widening the circle of people we should include.
We thought of ways of encouraging as many people as possible to take part in the review, such as:
People had volunteered to be on the steering group and others joined us later to help with the survey. There were several ways we could each contribute to the review. For example, different people were good at:
People gave us feedback about how useful they found the website and the IT training courses. We all gave ongoing feedback on how the review was progressing, any problems that were developing, and what seemed to be working well and what wasn't working so well. We all found ways to tell others about the project and get them involved. They included telling our families, friends and workmates, and getting information published in community newsletters, the local newspaper, and on the Shire website.
We had to work out what to include in the review. Statistics that we obtained from a recent Shire Council report on their IT training courses were useful in helping us work out what information we already had and what we still needed to collect. Because we decided that we wanted to do a survey, we needed access to printing, stationary and postage costs. The Council and schools helped us with this, as well as with word-processing the survey itself and printing it. For a useful review, we also needed to think about which people were likely to be willing and able to help with such things as:
The first step in doing our review was asking what we wanted from our IT projects. Why had the website been set up and what were our goals of the Community Futures Network website? Was the way we planned and designed it effective? What specifically did we think the website and the IT training courses would achieve for ourselves, for other people and groups, and for the community as a whole? We remembered that one of our goals for setting up the Westana Community Futures Network website had been to become better known in the community. Other people could see what we were aiming to achieve and become interested in contributing to helping create a vibrant community with a future. They could learn more about us from the links to our personal home pages, and contact us easily using the links to our email addresses. We thought this would encourage people to join in our activities, including our online discussions. The goals for the Internet training course in the local library were to help us become more skilled in using the Internet to efficiently search for information, to send and receive emails and attachments, and to create simple home pages. The goals for the computer basics course at the State School were to familiarise people with computers so that they would feel more comfortable and confident with using their home computers and the Internet. The aim was for participants to learn basic word processing, and how to create spreadsheets, format documents, manage files, and other basic skills. The long-term goals of these IT projects included developing skills and tools to help people in their work and leisure, to enable community groups to work together better, and to know more about and join in with community activities. We thought our IT projects could create more opportunities for learning, employment and business development. This could encourage people to stay in, or even move to, the thriving 'learning community' we envisaged.
Next we looked at how we had been trying to achieve these goals through our Community Futures Network website and our IT courses. What had we actually done and what were we doing? We noted down everything we had done, including all the major and minor things that went into the projects. There was too much to list here but, in brief:
But how well had we done? This was the main point of the review - what all the previous work had been building up to. So we wanted to do it well. How could we assess how well our IT projects had achieved their goals? This took several workshops to organise, and included:
David Cordell, the Community Development Officer, helped us run the workshops and focus groups. Members of our group searched the Internet and did some reading to learn more about developing surveys and holding focus groups. Dr Sally Green, a university researcher, gave us advice on the design of our survey and the focus group and interview questions. (see Suggested ways of using EvaluateIT on this website to get some idea of what's involved in running workshops and using IT to collect feedback and tell people about your review project. The Resources page on this website also has links to information sheets on methods such as brainstorming and focus groups, as well as useful toolkits and publications). We planned our survey, focus groups and interviews to find out:
When we put together everything we had learnt from the review, we were able to see what the main problems were, and which people and groups in our community were missing out. We also had a range of suggestions on how to improve both the Community Futures Network website and the IT training courses. Among other things, we found out that:
We had gathered many good ideas about things we could do to improve our IT projects, such as:
We had gathered plenty of interesting findings, but, before we acted, we needed to check them out:
We also wanted to think about what we had learned about doing reviews so that we could do the next review better. We held a workshop to look at:
We told as many people as we could about the review and what our main findings were, to try to get more of the community involved in the IT projects and thinking about how IT can benefit them and our community. We used several methods:
This publicity made more people aware of the review results and we received more feedback about the review. This included feedback from those involved in the Community Futures Network or who had done the IT training courses (or missed out), and many others in the community, both in town and on properties. We also started to get some interest from potential sponsors such as an Internet service provider. Most people commented that our review results were interesting and useful. They said it had increased their awareness of the opportunities that IT could provide and the ways that local IT resources could be better used. Councillor Park said she was pleased with the quality of the report and the very clear and effective way that the findings had been presented.
The next stage was to make the changes that we decided needed to be made, based on all the ideas and suggestions. This took several steps:
As we are carrying out these changes we are continuing to have meetings and to think about going through this review process again in the future. We are thinking of making more use of email and our website to gather feedback and holding teleconferences to link some remote people who cannot travel to our meetings. We know that next time, it will be easier and quicker because we will use the step-by-step process and questions on the EvaluateIT website to keep us focussed on what we are doing and why. We feel more confident in regularly reviewing our IT projects and we have learned a lot about making better use of our IT resources to help the community develop in the ways we want. We have all learnt new skills, not just in IT, but also in how to facilitate meetings and workshops, conduct surveys and analyse them, write reports, and speak in public about projects in the Westana community. The review process has created strong new connections between those involved. Some of the barriers between different groups are also starting to break down as more people talk to each other and find out what the different groups in our community are doing and how we can work together better. Note: Westana is a fictional community but parts of this story have been adapted from actual examples of rural communities reviewing their IT projects.
A regional community group reviews the impact of IT courses offered in their community by the BHP-Billiton Skillsnet Roadshow Queensland.
A small group of librarians, IT trainers, some past course participants, two council staff (an IT service desk consultant and a youth worker) get together to plan the review. It is now two years since the BHP-Billiton Skillsnet Roadshow Queensland visited their city. They want to find out how useful the courses proved to be and what follow-up (if any) is needed. Before they go ahead they discuss any risks that might be involved and how they plan to minimize them.
They decide they need to widen the group to involve staff representatives from several groups. These include the Town Council, and local IT businesses/industry partners who provided support for the courses. They work out ways to get these folk interested and involved. They also consider what resources, talents and contacts they themselves have to contribute, and what else the group needs to carry out an effective review.
First they make sure they agree on the goal for the review. They decide they all want to see 1) whether the courses met the needs of the local community, and 2) whether more courses are needed and if so, which ones. They discuss what the needs of the local community are and how they can investigate whether the Roadshow courses met these needs or not. They decide to look at statistics about how many people attended courses and which courses they attended. They also decide to carry out a survey to find out how useful the courses were for local people, and whether the associated course materials on the Roadshow website were being used. Then they work out how they will do this and who will do what. Finally, when they have their findings, they discuss what needs to be done to address the community's unmet training needs. They find that the earlier courses have created a huge demand for further IT training and that other groups in the community are both willing and able to help the local library meet this demand.
After checking that their survey methods are thorough and fair, they discuss how to let the rest of the community know, how to get feedback about what others think, and how to make the changes that need to be made.
A Learning Network Queensland (LNQ) Centre in a regional town in Queensland used EvaluateIT to do a review of how the services available matched up with the needs of users and would-be users. They went through the four steps (1. Plan the review, 2. Involve people, 3. Do the review, and 4. Review results and make the changes) and found that they could make much better use of the resources available by planning to accommodate the lifestyles of those who want to use the Centre. For example, they created a new booking system and set up a roster of local people who could be contacted when problems arose.
This email discussion list looked like a promising idea when a few Indigenous students got together to set it up. The idea was to have a support network for Indigenous students living away from their communities while they attended schools, TAFEs and universities in the city. However, it didn't really get off the ground. They decided to use the EvaluateIT resource kit to plan and carry out an evaluation of the email discussion list. They used the email list itself to share ideas as they went through the four steps (1. Plan the review, 2. Involve people, 3. Do the review, and 4. Review results and make the changes). In the process, they found that there was, in fact, a real need for such a list. They shared lots of great ideas about how to contact new students and have them use the support available on the list to sort out some of the teething problems they had as they settled into their new lives in the city and at their various educational institutions. It also became a place where notices for get-togethers and other events could be posted. |
Example 1: the Westana review of IT projects
Step 1: We plan the review 
Step 2: We involve people in the review
Step 3: We do the review 
Step 4: We review results and make the changes 
