Development of a Metric of Aquatic Invertebrates for Volunteers (MAIV): A Simple and Friendly Biotic Metric to Assess Ecological Quality of Streams

Abstract

Citizen science activities, involving local people in volunteer-supported and sustainable monitoring programs, are common. In this context, the objective of the present work was to develop a simple Metric of Aquatic Invertebrates for Volunteers (MAIV), including a user-friendly tool that can be easily accessed by volunteers, and to evaluate the efficiency of a volunteer monitoring program following an audit procedure. To obtain MAIV values, macroinvertebrate communities were reduced to 18 surrogate taxa, which represented an acceptable compromise between simplicity, efficiency, and reproducibility of the data, compared to the regular Water Framework Directive monitoring. When compared to results obtained with the National Classification System of Portugal, MAIV accurately detected moderate, poor, and bad ecological status. Thus, MAIV can be used by volunteers as a complement to the official monitoring program, as well as a prospective early warning tool for local problems related to ecological quality. Volunteers were students supervised by their teachers. Results obtained by volunteers were compared to results obtained by experts on macroinvertebrate identification to measure the efficiency of the procedure, by counting gains and losses on sorting, and identification. Characteristics of groups of volunteers (age and school level) did not influence significantly the efficiency of the procedure, and generally results of volunteers and experts matched.

1. Introduction

The European Water Framework Directive (WFD) instructed all member states to achieve good ecological status in all water bodies [1]. The process required the implementation of monitoring programs to assess the ecological integrity of each water body in a continuous manner. Within this context, bioassessment became relevant in ecological monitoring, and substantial effort was put into developing multimetric indices that would best express human pressures on ecosystems [2].

Bioassessment programs need qualified expertise to make key decisions in the field and to identify organisms in the laboratory. A rather interesting way to circumvent the low number of experts in ecological monitoring has been the development of citizen science (CS), that is, the science done in conjunction with volunteers, under the direction of professionals and scientific institutions [3]. In addition to the benefits for science, CS produces important societal benefits through innovative thinking and by improving: (a) societal conditions through scientific outreach; (b) education for the promotion of conservation [4]; and (c) relationships among institutional actors, bringing together individuals from diverse backgrounds [5].

CS activities involving local people in volunteer-supported and sustainable monitoring programs is a common practice [6,7,8,9,10,11,12]. For example, in Canada, both federal and provincial governments have initiated aquatic biomonitoring networks such as the Ontario Benthic Biomonitoring Network (OBBN), accessible to both volunteers and scientists [3]. In general, CS has been used in hydrological observations and/or in the monitoring of surface water quality in all different continents of the globe. This is the case of Europe [13], North America [14], Central America [15], South America [11], Oceania [16], Asia [17], and Africa [18], demonstrating that CS is a low-cost and crucial tool in raising awareness on the importance of good water quality [13], and important in decision-making instances [19]. When educators are involved in CS, the benefits for school environmental education are multiple [20,21]. The improvement of student learning skills and the professional development of science teachers through engagement in scientific inquiry and research, are two of such benefits. These ultimately improve literacy in Science, Technology, Engineering, and Mathematics (STEM). STEM is defined as the ability to read and/or write science texts, tables, and graphics, and the derived skill to apply scientific knowledge [22].

The integration of results coming from CS volunteer programs as a part of the National Classification System (NCS) must be used carefully, because potential conflicts might occur between environmental authorities and volunteers [23]. As an attempt to prevent such conflicts, certain compromises are required [24], for example: (a) water authorities need to have valuable and precise results to support management actions, but volunteers prefer more friendly tools that are not necessarily precise; (b) water authorities desire results to accomplish legislative requirements, whilst volunteers are most likely driven by self-interest (hobbies, leisurely pursuits) [20,25]. Thus, a participative balance has to be reached in choosing the sites to be monitored [24], and an equilibrium must be set between friendly tools and realistic results, quick results and valuable data, and inexpensive and sufficiently precise results. The challenge is to come up with the friendliest tool for volunteers to produce useful information for management proposals. For this, water authorities should know how volunteer results differ from expert results, because any metric of ecological quality is of little use without an understanding of the uncertainties in its estimation [26,27,28,29].

To develop a supported and sustainable volunteer monitoring program, it is necessary that volunteers have grasped the basic scientific concepts through adequate literature or through training courses [30]. To prevent errors, sampling and laboratory protocols need to be simple and standardized. Finally, an expert audit should be implemented to assess the quality of the results, detecting main gaps and improvements to be made, thus increasing the efficiency of the process.

Macroinvertebrates are probably the most popular biological quality element evaluated by volunteers on lotic ecosystems [29]. Macroinvertebrates are easy to collect with simple equipment and to identify with the naked eye in the field. They colonize all aquatic habitats and they have a great diverse sensitivity to physical, chemical, organic, and morphological pressures. However, the general metrics and multimetric indices developed to assess ecological status based on macroinvertebrates require counting and identification of, at least, all sampled families or even the identification of some species [2]. The amount of work and time required for these identifications could be annoying to volunteers, increasing the risk of low accuracy and desistance. Thus, to maintain the interest of the volunteers, it is necessary to simplify the metrics through a reduction of the taxonomic detail [29] and to reduce the counting process, thus obtaining quick results but maintaining the accuracy.

The starting point of the present study was a set of tools developed by the European project on Conservation and Sustainable Development of Freshwater Ecosystems (CONFRESH, https://www.nhmc.uoc.gr/en/museum/programs/1814) to assess stream ecological quality based on macroinvertebrate communities. These tools were applied in a regional project conducted by the Regional Hydrographic Administration of Algarve of the Portuguese Environmental Agency (APA-ARHAlg). The objective was to evaluate the accuracy of the results obtained by students from 5th to 12th grades (volunteers), during extracurricular activities supervised by their teachers and following a participatory/contributory model of participation [31]. The following steps were used for this activity: (1) APA-ARHAlg selected study sites, (2) teachers were taught skills on stream ecology and monitoring during courses given by experts, (3) teachers and their students (volunteers) collected and examined the samples and determined on their own the ecological class of each sampling site, and (4) experts on macroinvertebrate taxonomy audited the results. During the audit procedure, experts reanalyzed samples and quantified the differences between the two analyses, expressing them as the accuracy of the results obtained by volunteers. It was expected, as a hypothesis, that the characteristics of volunteers (age, number of students, and number of teachers) could influence the accuracy of the results and, in opposition, that this accuracy is independent from the ecological quality.

We intended to test if simple tolerance metrics based on the reduction of the more than 120 macroinvertebrate families to just 18 surrogates (Class, Order, and Family), as proposed by the CONFRESH project, did not significantly compromise the quality of the biological assessment when compared to the results of the National Classification System of Portugal. Thus, the aim of this paper is to propose a simple Metric of Aquatic Invertebrates for Volunteers (MAIV) through a reduction of the taxonomic detail, eliminating the counting process but maintaining the accuracy. We propose MAIV as an upgrade of the metric calculated by the CONFRESH project, since its score is the sum of tolerance scores of all surrogates (summative metric) and not only the tolerance score of the most sensible surrogate (nonsummative metric), thus increasing accuracy.

2. Materials and Methods

A synthetic overview to follow the methodological procedures adopted in this study is presented on Figure 1.

2.1. Study Area and Sampling Period

The study was developed in the Algarve region (southern Portugal), characterized by Mediterranean climate conditions, where the aquatic environment is shaped by sequential changes of annual flooding and drying, directly affecting the physical and biological characteristics of the streams [32]. Most of the streams are temporary, and urbanization, agriculture, and cattle raising are the most important human pressures. From 17 streams, selected by APA-ARHAlg in this region and covering the four river types presented in Algarve (southern rivers with medium to large dimensions, drainage area more than 100 km² (M–L); southern mountainous siliceous rivers (M–S); southern small siliceous rivers (S–S), drainage area less than 100 km²; and calcareous rivers of Algarve (C)), 36 samples were collected during spring 2010, after a period of intense precipitation (Figure 2).

2.2. Teacher Training Courses and Volunteer Skills

Skills to implement this volunteer monitoring program were developed in six similar teacher training courses on “Conservation and Sustainability of Freshwater Ecosystems”, covering the entire Algarve region, between January and May 2010. These courses were supported by bibliography on aquatic ecology, field and laboratory procedures, and macroinvertebrate identification [33,34], developed by the CONFRESH project. Additional information regarding the WFD and basin management was provided [1]. The training courses were organized into four sequential parts: (1) theoretical lectures on aquatic ecology, basin management, and monitoring; (2) field activities preceded by laboratory practical lectures; (3) question-and-answer sessions to solve doubts and to help with macroinvertebrate identification to family level, and (4) presentation of results. The institutions collaborating with the courses were the six Teacher Training Centers, Regional Hydrographic Administration of Algarve of the Portuguese Environmental Agency (APA-ARHAlg), Algarve Educational Authority, University of Évora, and University of Algarve.

In total, 87 teachers attended the courses and applied/transferred the obtained skills to their 807 students distributed in 23 secondary schools. This was done during extracurricular activities, forming 36 volunteer groups (students under teacher supervision). The students were from 36 classes with sizes varying from 4 to 66 students per class, degrees from 5th and 7th–12th, ages from 11 to 16, and number of teachers from 1 to 3 teachers per class (Table S1). None of the students had previous experience with macroinvertebrate assessment.

2.3. Habitat Description and Composite Human Pressure Gradient Calculation

Site habitat description was done by filling a form in the field. This form was produced by the APA-ARHAlg, covering four main categories of descriptors that were categorized as presence/absence on both banks of each river reach (land use, settlements on the river, water status, and river status) (Table S2).

Based on the results obtained from site habitat description, a Multiple Correspondence Analysis (MCA), applied to a matrix of the 36 samples described by presence (Yes)/absence (No) of 32 descriptors (Table S2), covering land use (11), settlements on the stream (7), state of the water (11), and stream characteristics (3) was performed, and the composite human pressure gradient established by the scores of the samples on the first dimension of the MCA.

2.4. Macroinvertebrate Sampling and Laboratory Procedures

In a 50-m-long reach at each site, the percent cover of the aquatic habitats was estimated (boulders, stones, sand, silt, macrophytes and algae, and coarse particulate organic matter—CPOM), following the procedure adopted by the Portuguese monitoring program [35]. The composite sample consisted of six kick subsamples (each 1 m in length) collected with a hand net (0.5 mm mesh size, 25 cm front) (Figure 3a). These six kick subsamples were distributed along the reach covering different habitats, according to their percent cover. Each composite sample was sorted live in the field or immediately fixed after collection with formalin, kept in plastic flasks, and transported to the laboratory.

Macroinvertebrates were sorted and identified into 18 surrogates (Table S3), following the teaching materials developed by CONFRESH [33,34] (Figure 3b,c) and complemented by other more complex keys [36]. In order to simplify the identification process, a key was developed to identify the 18 surrogates (see Figure S1; Tables S4 and S5). The ecological quality class for each sample was the score of the most sensitive taxa present in the sample (Figure 3d; Table S3). All macroinvertebrate sampling and laboratory procedures were done by volunteers (students under constant supervision by teachers).

2.5. Audit Procedure

The objective of the audit procedure was to quantify the accuracy of the results obtained by the volunteers. To attain this objective, the volunteer results (primary analysis), were checked by experts (auditors) from the universities of Évora and Algarve. The accuracy of the volunteer results was measured by the quantification of the differences between primary and audited analysis, following an adaptation of the audit procedure implemented during the European STAR (Standardisation of River Classifications) project [37].

The primary analysis consisted of: (1) sorting the organisms in each sediment sample, (2) identification down to the established 18 surrogates (Table S3), and (3) establishment of the respective water quality class (Table S3). Once the primary analysis was completed, the remaining sediment from each sample was kept in plastic flasks and preserved in 70% alcohol. Likewise, the macroinvertebrates identified by the volunteers were kept in individual vials and preserved in 70% alcohol. Flasks and vials were labeled accordingly, affixed with the respective taxa list (primary taxa list) and ecological quality class (primary classification), and sent to the auditors.

Experts (auditors) checked the sorting and identification done during the primary analysis, termed the sorting audit and identification audit, respectively. On the sorting audit, the auditors re-sorted the sediment of each sample, detecting the remaining surrogates not collected during the primary analysis. Every surrogate (Table S3) found by the auditors and not detected in the primary analysis was considered a gain of the sorting audit. The sorting audit, for each sample, was quantified by the respective number of gains.

For the identification audit, the auditors checked the identifications of the primary taxa list. Two situations were possible during the identification audit: (a) gains of the identification audit were recorded if a new surrogate not previously found by the primary analysis was identified; and (b) losses of the identification audit were recorded when individuals of a given surrogate were allocated into different surrogates, thus increasing the number of surrogates present. The result of the identification audit was quantified by the number of gains and losses. The integration of the sorting and identification audits corresponded to the total audit. After the audit procedure, a new taxa list was obtained (audited taxa list), and the respective ecological quality class recalculated (audited classification).

Spearman correlations between the results of the total audits and volunteer characteristics were calculated to detect possible effects of the latter (Table S1) on the accuracy of the volunteer assessment. Spearman correlations between total audit results and water quality were also calculated to test the independence between the accuracy of the volunteers and water integrity. For these last correlations, water integrity was measured in two ways: (1) ecological quality class obtained by the audited classification, and (2) the composite gradient of human pressures obtained by the scores of the samples on the first dimension of the MCA.

2.6. Accuracies of Simple New Tolerance Metrics Compared to the NCS

In order to select an accurate and simple tolerance metric for macroinvertebrates, two kinds of metrics, based on the 18 surrogates were tested: (1) the CONFRESH metric [33], which score (value ranging from 2 to 5) is that of the most sensitive surrogate present in the sample. This score is a very simplistic picture of the ecological state of the site (nonsummative metric—NS); and (2) a set of three metrics, which final score is the sum of the tolerance scores of all surrogates present in the sample (summative metrics—S1, S2, and S3). These added scores give a better and more accurate depiction of the ecological state of each site. For S1, all surrogates included in the same group of tolerance had the same tolerance score adopted by the CONFRESH metric, ranging from 5 (the most sensitive surrogate) to 2 (the most tolerant surrogate). For S2, the tolerance score of each surrogate was based on the mean of the family tolerance scores established by the Iberian Biological Monitoring Working Party (IBMWP) [38] (e.g., the Trichoptera tolerance score is 8.5, corresponding to the mean of the tolerance scores established by IBMWP for its respective 20 families). In this S2 metric, all surrogates of each group of tolerance have the same score calculated as the mean score of all their respective surrogates. At S3, surrogates from each group of tolerance have different scores, according to the mean of their taxa scores. All these mean scores were rounded to the nearest natural number. A synthetic overview of the tolerance scores of each surrogate included in each group of tolerances is presented in

Table 1. Tolerance scores established for surrogates for the different tested metrics. The nonsummative (NS) metric score corresponds to the highest surrogate score. Scores of the other metrics (S1, S2, S3) are obtained by the sum of the scores of all the surrogates present. The Mean IBMWP column shows the mean tolerance score attributed by IBMWP (Iberian Biological Monitoring Working Party) to families included in the surrogate. The number of families included in each mean are shown in parentheses.

Taxa	NS	S1	Mean IBMWP	S2	S3
Oligochaeta	2	2	1.0 (1)	2	1
Chironomidae		2	2.0 (1)	2	2
Planaria	3	3	5.0 (3)	5	5
Hirudinea		3	2.5 (4)	5	2
Gastropoda		3	4.3 (11)	5	4
Bivalvia		3	4.5 (2)	5	4
Isopoda		3	6.0 (2)	5	6
Anisoptera		3	7.5 (5)	5	7
Zygoptera		3	6.7 (3)	5	7
Coleoptera		3	3.8 (13)	5	4
Simuliidae		3	3.0 (1)	5	3
Diptera		3	4.2 (19)	5	4
Amphipoda	4	4	6.0 (3)	6	6
Crustacea		4	4.5 (2)	6	4
Ephemeroptera		4	7.5 (11)	6	7
Trichoptera		4	8.5 (20)	6	8
Megaloptera		4	4.0 (1)	6	4
Plecoptera	5	5	9.6 (7)	10	10

. For each summative metric (S1, S2, and S3), a similar procedure was adopted to establish the boundaries between ecological classes: (1) the highest possible score was calculated, adding the scores of all surrogates (59 for S1, 94 for S2, and 88 for S3); (2) the boundary between high and good classes was established as 35% of the maximal possible value of the score (approximately the lower level of the boundary of high–good classes established for the Portuguese monitoring program [35]); (3) the remaining 65% was divided into four equal intervals to establish the boundaries between the other ecological classes, following the procedure of the Portuguese monitoring program [35]; and (4) the boundaries were rounded up to the nearest natural number to prevent decimal numbers (

Table 2. Score limits of the ecological quality classes for each tested metric.

Quality Classes	NS	S1	S2	S3
High	4	37 to 59	61 to 94	61 to 98
Good	3	28 to 36	46 to 60	46 to 60
Moderate	2	19 to 27	31 to 45	31 to 45
Fair	1	9 to 18	15 to 30	15 to 30
Bad	1	1 to 8	1 to14	1 to 15

).

To evaluate the accuracy of these tested metrics, the APA-ARHAlg provided the data set of the National Classification System (NCS) for the study area, consisting of 79 sites with the official classification already established (NCS classification), as well as its respective taxa list (NCS taxa list). For all 79 sites, classifications of the four tested metrics (NS, S1, S2, and S3) were calculated (metric classifications), based on the NCS taxa lists. NCS classification and metric classifications were compared, and their respective accuracies measured by the number of correct/incorrect classifications, with the NCS classification as the benchmark. These comparisons were done for ecological classifications based on five ecological quality classes (high, good, moderate, poor, and bad), and on two ecological quality classes (high + good and moderate + poor + bad). This was to comply with the WFD target, which is to attain at least the good ecological status. These comparisons were done to all sites and again separately for each river type (southern rivers with medium to large dimensions, drainage area more than 100 km² (M–L); southern mountainous siliceous rivers (M–S); southern small siliceous rivers (S–S), drainage area less than 100 km²; and calcareous rivers of Algarve (C)) [35].

2.7. Metric Accuracies Under a Real Volunteer Contest Procedure

The accuracy of the four tested metrics (NS, S1, S2, S3) was also assessed under a real volunteer contest procedure, because the selected metric had to be used by volunteers. To quantify their accuracies, primary and audited classifications were calculated based on the respective primary taxa lists and audited taxa lists. Primary and audited classifications were compared, and their accuracies measured by the number of correct classifications, assuming the audited analysis as the benchmark.

3. Results

3.1. Gradient of Human Pressures

The ordination obtained by the MCA shows a clear increasing gradient of human pressures along the first dimension (Figure 4). An opposition is evident between nondisturbed and disturbed sites (Figure 4a) and between land use, e.g., forest versus agriculture (mainly explained by dimension Dim1, approximately 46% and 37%, respectively) (Figure 4b). Concerning the existence of settlements along the streams, an increase of impact intensity is observed along the first axis, from small dams and springs to houses, roads, and irrigations (mainly explained by Dim1; approximately 46%) (Figure 4c). An opposition is also evident between water state (Dim1 = 27.2%) (Figure 4d) denoting a clear relationship with the ecological status, and for this reason, considered the composite gradient of human pressures.

3.2. Audit

The audit procedure applied to the 36 samples collected by volunteers showed that gains and losses are mostly associated with identification (

Table 3. Gains and losses (totals and means per sample) for the three audits (sorting, identification, and total). A total of 36 samples were audited. Standard deviations appear inside parentheses.

Gains and Losses	Sorting Audit		Identification Audit		Total Audit
	Total	Mean	Total	Mean	Total	Mean
Gains	1	0(0.2)	100	3(2.6)	101	3(2.6)
Losses	0	0(0.0)	21	1(1.0)	21	1(1.0)

). It is also evident that there was a higher number of gains during the entire audit procedure as compared to the losses (Table 3). Concerning the identification done by volunteers, there were gains practically in all surrogates, with Diptera, Ephemeroptera, Coleoptera, and Plecoptera being the highest contributors (

Table 4. Gains and losses of the identification audit, distributed by the surrogates. The vernacular names of the taxa are inside parentheses.

Taxa	Gains	Losses
Planaria (flatworm)	0	0
Oligochaeta (worms)	7	0
Chironomidae larvae (midge blood larvae)	9	0
Simuliidae larvae (black-fly larvae)	8	0
Diptera larvae (fly larvae)	9	2
Hirudinea (leach)	3	1
Gastropoda (freshwater snail)	9	0
Coleoptera adults and larvae (beetle)	10	3
Bivalvia (freshwater mussels)	0	0
Zygoptera larvae (damselfly)	6	0
Anisoptera larvae (dragonfly)	5	0
Isopoda (freshwater slater)	0	0
Amphipoda (freshwater scud)	0	0
Trichoptera larvae (caddisfly)	6	1
Ephemeroptera larvae (mayfly)	14	0
Crustacea (shrimps and crabs)	4	0
Megaloptera larvae (alderfly)	0	0
Plecoptera larvae (stonefly)	10	0

).

Results included in

Table 5. Pearson correlation of the total audit (gains and losses) with the volunteer group characteristics, the human pressures (composite gradient), and the quality classes.

Gains and Losses	Scholar Degree	Age of Students	Number of Students	Number of Teachers	Human Pressures	Quality Classes
Total gains	0.10	0.04	−0.22	0.24	0.63 **	0.07
Total losses	−0.12	−0.22	0.14	−0.17	−0.330 *	−0.14
Number	36	33	35	37	36	37

* p < 0.05; ** p < 0.01.

showed no significant correlations in terms of gains and losses with the characteristics of the groups of volunteers (p > 0.05). The only significant correlations (p < 0.05) were obtained for human pressures, expressed as the scores of the samples along the first MCA dimension (Figure 4).

For the great majority of the sites (22), primary and audited classifications match closely (

Table 6. Comparison of primary and audited classifications. Underevaluation occurs when primary classification is lower than the audited classification, with the opposite for overevaluation. When both classifications match, it is considered as correct.

	Underevaluations		Correct	Overevaluations
	Two Classes	One Class		One Class	Two Classes
Primary to Audited	4	8	22	0	0

). When classifications do not match, primary classifications were underevaluated in one class (eight cases) or in two classes (four cases).

3.3. Accuracies of Simple New Tolerance Metrics Compared to the NCS

All metric classifications are significantly correlated (p < 0.01) with the NCS classification (

Table 7. Spearman correlations between the National Classification System (NCS) classification and metric classifications (** p < 0.01).

National Classification System	Metrics
	NS	S1	S2	S3
NCS	0.496 **	0.548 **	−0.58 **	0.60 **
Number of sites	79	79	79	79

). However, a more detailed analysis, based on the incorrect classifications for five and for two quality classes, gave different results. The NS shows a higher tendency for overevaluations than S1, S2, and S3 (Figure 5). The exception occurs with the southern mountains river type without any unimodal pattern, showing a relative minimum for the correct classifications. However, results for the southern mountains and the southern medium–large river types are based on a lower number of sites. Similar results occur when only two ecological quality classes are considered (graphs in the right column of Figure 5).

No more than 18% of the classifications obtained by S1, S2, and S3 for the two ecological quality classes, assuming an uncertainty of 5% around the boundary (Figure 6a–e), are incorrect. Once again, the river types with the lowest number of sites (south mountains and large–medium river types) do not fit into these trends.

3.4. Metric Accuracies Under a Real Volunteer Contest Procedure

The comparison of primary classifications and audited classifications, for the four tested metrics (NS, S1, S2, S3) and on the 36 sites collected by the volunteers (

Table 8. Comparison of quality classes, between primary and audited analyses of 33 samples analyzed by the volunteers for the tested metrics (NS, S1, S2, S3). Underevaluation occurs when the class quality of the primary analysis is lower than the audited analysis; an overevaluation occurs in the opposite case. When both classifications match, it is considered as correct. Three samples were not included due to deficient preservation of the material sent to the auditors.

Tested Metrics	5 Quality Classes			2 Quality Classes
	Under	Correct	Over	Under	Correct	Over
S1	13	19	1	3	28	2
S2	13	17	3	2	30	1
S3	15	16	2	2	29	2
NS	11	22	0	10	23	0

), shows that the number of correct classifications tends to be higher than the underevaluations and the overevaluations. When five quality classes are considered, the number of underevaluations is very close to the correct evaluations, but when two classes are considered, the correct classifications compose the great majority of the summative metrics (S1, S2, S3), showing that NS has the lowest value of correct classifications (Table 8).

4. Discussion

Evaluating the accuracy that volunteers have in determining the ecological quality class of a site using macroinvertebrates comprises four main components: (1) sampling, (2) sorting, (3) identification, and (4) acceptability of the final results (assignation to an ecological quality class). The sampling procedure, if well standardized, seems not to be a significant source of error [28]. Sorting seems to be the most interesting phase to volunteers. The curiosity stimulates the demand of new (different) macroinvertebrates and a constant attention is devoted to this task, this being a possible reason for the absence of gains on the sorting audit. The most important source of error occurred during identification, leading to some gains and to a few losses. Gains probably resulted from two different sources: (1) error on the identification of morphologically similar surrogates (e.g., Ephemeroptera and Plecoptera, Diptera and some larvae of Coleoptera, or Trichoptera) and (2) confusion with grains of gravel (e.g., Gastropoda) or small plant pieces (e.g., Oligochaeta). In any case, mean gains per sample were not too high. Thus, it can be concluded that these results are acceptable and can probably be enhanced with more experience and adequate materials to support the identification process [39].

Results showed independence of gains and losses from the characteristics of the groups of volunteers, thereby rejecting the initial hypothesis. Then, the adopted procedure of sorting and identification can be applied to a wide universe of possible volunteers. However, number of gains and losses seem to be dependent on human pressures, thus rejecting the hypothesis of independence. The most degraded sites tend to be dominated by a small number of more tolerant taxa [40], making it difficult to single out the few different ones; therefore, more gains were detected (positive significant correlation, Table 5). In contrast, sites with better quality tend to have richer communities [40], and individuals belonging to the same surrogate can be distributed by several surrogates. However, those gains and losses did not affect the ecological quality assessment, because the number of correct classifications (the same between primary and audited analyses) outweighed the incorrect ones, a fact that confirms other studies done with experts [37] and nonexperts [29,41].

Previous studies also assessed the reliability of stream monitoring by volunteers in relation to professionals (e.g., [6,16,26,27,29,42,43]) and most of them concluded that, with appropriate resourcing and robust protocols, volunteer data closely agree with the professional data used by government reporting and decision making. High correlations were also obtained between primary and audit analyses in New Zealand for %EPT—Ephemeroptera, Plecoptera and Trichoptera [16] and in the U.S.A. by the volunteer programs in Virginia [43] and Seattle [26].

Although significant correlations were obtained between scores of the four tested metrics and the NCS, the three summative metrics gave better results when the quality classes were compared. This confirms the hypothesis of a better performance of the summative metrics. Generally, the NS tended to overevaluate the ecological assessment because it reduces information to only the most sensitive taxa, independently from community composition. The summative metrics accounting for the tolerance scores of all the surrogates present were found to balance taxa richness and tolerances. Reducing the five quality classes of the WFD to two classes, separated by the good–moderate boundary, showed that the number of correct classifications increased, making these summative metrics more acceptable to classify water bodies that are below the good quality status (the environmental target of the WFD).

The accuracy of the volunteer procedure (difference between primary and audited analyses) is very similar to the tested metrics (summative and nonsummative) when five ecological quality classes are established. However, if only two quality classes are established (boundary good/moderate), the accuracy of the summative metrics is higher than the nonsummative one.

Since the results obtained by S1, S2, and S3 were very similar and more accurate than those obtained by NS, S2 was selected to be the Metric of Aquatic Invertebrates for Volunteers (MAIV) for two main reasons: (1) the tolerance scores of the surrogates are based on published results [38] and (2) the scores of all surrogates included in each of the four groups of tolerance are equal, thus, easier to use. A detailed calculation protocol of the MAIV is provided in Tables S4 and S5.

The MAIV proved to be adequate for volunteers, because it is based on a set of a few surrogates easily recognized by nonexpert people, saving time when compared to other volunteer monitoring experiences in which the identification is done to the family level [26]. In addition, its application is independent from the characteristics of the volunteers, and the determination of the ecological quality class is simple and quick, keeping volunteers interested. However, the use of MAIV, to complement the NCS raised some concerns that need to be pointed out. In fact, MAIV reduces the assessment to one tolerance metric, while the NCS [35] is based on an index composed by metrics of tolerance, composition, and diversity. This is the reason for the lower accuracy of MAIV in arranging water bodies into five ecological quality classes. However, MAIV still is a good tool for the detection of water bodies that are below the good ecological quality status (the WFD environmental target).

The use of single volunteer data (MAIV classifications) for management purposes can be risky, due to the lower precision. However, its use integrated with a set of samples to detect geographical and/or temporal environmental patterns can be an important complement for water authorities, as already pointed out by Deutsch et al. [9], saving time and costs [30,44,45]. Thus, to prevent incorrect management decisions based on MAIV, it is advisable to keep an interval of 5% of uncertainty around the good–moderate boundary (scores 41 to 49).

The results obtained in the present exercise show that MAIV can be applied to different Portuguese river types with the same boundaries, with acceptable results, like the south limestone and south small rivers types. But no definitive conclusions can be reached concerning the other two river types, due to the reduced number of sites used in the exercise. However, if the MAIV will be used in a first step by volunteers, it had to be kept simple. Different boundaries for different river types could introduce a complexity not important for the objective of MAIV. This latter metric could provide an extensive coverage of river assessments, complementing the NCS. Thus, the obtained results by MAIV could act as a warning of human pressures on river sites that are not regularly assessed by the NCS, indicating where more detailed monitoring or management actions could be needed.

The involvement of the educational community in the training courses proved to be quite effective. Teachers had the chance to develop their professional profile by improving the STEM literacy, through theoretical lectures that related to practical aspects (i.e., field and laboratory practical lectures) on conservation and sustainability of freshwater ecosystems. Some other benefits of this study were student citizenship, connection of knowledge, awareness and behavior, and partnering student biomonitoring research with community needs, as well as those of stakeholders and policy makers. In general terms, volunteers are strongly encouraged by interaction with scientists, and identified learning as one of the main rewards of taking part in CS projects, as documented by other studies [46,47,48]. The training procedure confirmed that SC, providing only theoretical information, is insufficient to elicit the required behavioral changes, with the training on identification being a key factor to the success of volunteer monitoring programs [27,49]. But identification should be complemented with other tools [11,42,43] such as simplified identification keys [27]. Hence, if it is desired to elicit new perspectives on how the behavior of individuals connect them to services provided by their ecosystem, and if the premise that “individuals act ethically as an integral part of an interconnected society and biosphere” is to be accomplished, it should be reached through personal belief rather than rational understanding [50].

Although there has been an increase in the number of studies on environmental monitoring research, precise tools to measure data quality are still scarce [25,51], compared to other volunteer practices, such as those in the health sector, for example [25]. More effort is required to evaluate data quality and to adjust more specific and simple methodologies, this to give credibility to volunteer data and enhance the effect on environmental community education [25]. A compromise is needed, between less uncertainties in environmental data and environmental community education, supporting volunteers in a balanced way.

The dimension of the regional project reported herein, involving six teacher training centers, is too large to be repeated, mainly due to the limited available resources (time and budget). However, the skills obtained by citizens was the seed for the creation of several small groups of volunteers (environmental school clubs involving senior citizens) that are being trained specifically in sampling procedures and identification; a much lower effort than organizing six entire training courses.

5. Conclusions

After training, students are able to identify aquatic invertebrates using simplified invertebrate keys and indices, a skill that can be used to accurately detect ecological status lower than good. Protocols and methods used for the monitoring of streams proved to be well accepted by volunteers. Voluntary monitoring as well as economic viability may contribute to social perceptions regarding environmental issues (e.g., [11]). The MAIV proved to be a good compromise between simplicity and result acceptability. Results show that the MAIV does hold up as a method for citizen scientists to use and that compared to the use of the same method by professionals, it is quite appropriate for volunteers. MAIV, used in a first step by trained volunteers, can be a valuable complement of the official monitoring program, adding value to the detection of possible ecological problems in sampling sites. The audit procedure showed a good performance by volunteers, with results not being significantly different from those obtained by experts. Concerning the evaluated metrics, summative metrics (sum of the scores of all present surrogates) are more accurate than the nonsummative metric, which final score is the highest surrogate tolerance score.