MANAGING BASIC EDUCATION
ASSESSING THE IMPACT OF THE PROGRAM ON STUDENT PERFORMANCE
1. Introduction
1.1. An Outline of Assessment Procedures
The ultimate success of the MBE program must be assessed in terms of the impact on students. However student performance is complex, as is its assessment and as it encompasses knowledge and understanding, skills and attitudes. The national school examination (UAN ) and half-yearly tests are limited in their nature mainly to factual recall of knowledge and not comparative from year to year or between different geographic areas. The program has, therefore undertaken its own student performance evaluation in Bahasa Indonesia, Mathematics and Science. The assessment was matched to the objectives of the teacher training program and the new competency based curriculum and focuses both on the development of student skills and knowledge which is understood and both of which can be applied.
|
The tests used were based on those developed under the World Bank PEQIP and Basic Education Projects and subsequently also used in the CLCC program. The tests consisted of:
a simple reading test for class 1 (administered orally with individual children)
a Bahasa Indonesia test in class 4 including a reading comprehension test and an essay question to assess writing abilities
a mathematics test in class 4, with an emphasis on problem solving skills
a science test in class 5 with an emphasis on the ability to interpret data and apply scientific knowledge
The tests were administered between 17 and 21 May 2004 in six schools in each of the ten districts taking part in the first and second phases of the program. This is a total of 60 schools, 30% of the total program schools, and approximately 40% of the primary schools. The schools tested included inti and imbas schools, SD and MI . In most districts three schools were chosen from each of the two clusters targeted by the program (each in a different subdistrict). In the case of Kota Batu, Kabupaten Probolinggo and Kota Madiun where the program is working in three subdistricts, two schools from each of the three clusters, one in each subdistrict, were chosen. A list of schools is attached in Annex 1.
It is planned to administer the tests again in the same schools in the same grades later in the project in order to assess changes in student performance as a result of the training and other interventions the schools have received. These tests are compatible both with the revised 1994 curriculum and the new competency based curriculum currently being introduced in 2004 in certain grades. The written tests were developed to take not more than an hour each. The Bahasa Indonesia, Mathematics tests were conducted with half of the relevant class, while the Science test was conducted with a maximum of 25 randomly selected pupils per class. The class 1 reading test was conducted with 12 randomly selected children in each school.
|
1 Ujian Akhir Nasional
2 Primary Education Quality Improvement Project (1992 - 98)
3 Creating Learning Communities for Children (UNESCO-UNICEF)
4 A 'inti' schools is the core school at the centre of a school cluster and the other schools are called 'imbas' or satellite schools.
5 SD (Sekolah Dasar) are conventional primary schools and MI (Madrasah Ibtidayah) are religious primary schools. Both are obliged by law to teach the same national curriculum.
6The children tested later in the project will be those then in the appropriate grade, i.e. for Bahasa Indonesia the current group of children in Grade 4.
7Discussions have been taking place as to whether there will be annual testing or only at the planned end of the project (in approximately three years time).
1.2. How the Results are presented
The results of these tests are discussed in sections 2 - 5 below for each test separately. The overall average score is given and comparative scores differentiated for (i) boys and girls, (ii) those who have attended kindergarten (TK) and those who have not, (iii) children from core cluster ('inti') schools and those from satellite ('imbas') schools and those attending conventional primary schools (SD) and those attending religious primary schools (MI).
Various charts are provided which (i) compare the average performance of students in the individual schools taking part, and (ii) compare the average scores of individual students. An analysis of the scores on individual questions is presented in the mathematics and science tests to determine which questions students found especially difficult.
When the tests are repeated the results will be compared to determine changes in performance since the first tests took place. It needs to be stressed that only six schools in each district were included in the test and are not necessarily a representative sample the districts' schools. Results are, therefore, not analysed and reported by district, but by school and pupil without specific reference to schools or districts.
Copies of the test have not been included with this report in order to avoid their inadvertent dissemination to schools which would make their further use invalid , as it is intended that they will be used again in the repeat testing.
2. Summary of Results
2.1. General
The schools tested in the ten districts included 49 conventional primary schools (SD) and 11 religious primary schools (MI). These included 20 inti schools (cluster core schools) and 40 imbas schools (satellite schools). Below in table 1 is a summary of the test results for each of the tests.
Table 1: Summary of Test Results for all Tests
|
|
Class 1 Reading Test |
Class 4 Reading
Comp. Test
(Max. Score 28) |
Class 4
Writing Test
(Max. Score 20) |
Class 4
Mathematics
(Max. Score 24) |
Class 5
Science
(Max. Score 38) |
Test 1
(Max. 20) |
Test 2
(Max. 5) |
No. of Pupils |
715 |
994 |
994 |
982 |
1203 |
% attended TK |
92% |
90% |
90% |
89% |
87% |
Boys |
16.52 |
2.74 |
14.62 |
10.57 |
15.07 |
17.76 |
Girls |
18.03 |
3.26 |
15.71 |
12.34 |
15.20 |
17.76 |
TK |
17.65 |
3.08 |
15.35 |
11.66 |
15.34 |
18.22 |
Non-TK |
13.12 |
2.15 |
13.66 |
9.96 |
13.45 |
15.92 |
SD Inti |
18.28 |
3.33 |
15.61 |
12.22 |
15.40 |
18.38 |
SD/MI Imbas |
16.80 |
2.83 |
14.95 |
11.05 |
14.98 |
17.44 |
SD(Conventional- Schools) |
17.47 |
3.08 |
15.25 |
11.58 |
15.18 |
18.13 |
MI (Religious Schools) |
16.50 |
2.60 |
14.91 |
10.93 |
14.90 |
15.95 |
All (Average) |
17.30 |
2.99 |
15.19 |
11.46 |
15.13 |
17.76 |
In general the scores were relatively evenly distributed in the mid range, with the exception of the reading tests, where scores in test 1 were clustered at the high end, suggesting that most class 1 children tested had mastered the basic skills of decoding words.
Performance did not differ greatly between SD and MI (conventional and religious schools) tested. This is contrary to experience in half yearly tests and the UAN where MI perform significantly worse on average than SD. However, it is probably due to the selection of MI in the MBE program. Most of the eleven MI tested were either state MI or supported by large foundations and all were chosen because of their ability to benefit from the program and show an example to other MI. This means that larger more viable and better performing schools were chosen.
The test results were very much higher than when the same tests were done in CLCC schools in February 2003. The tests may have been set in some cases at too low a level for most MBE schools (especially class 1 reading), whereas they may have been at too high a level for many CLCC schools. A number of factors are involved including the fact that all of the MBE schools are on Java, whereas two thirds of the CLCC schools tested were outside Java some in remote areas such as the outer islands of NTT and in Papua . A comparison of some of the results of the MBE and CLCC tests are set out below in section 2.6.
8 Unfortunately national tests, such as the UAN do not compare different areas of the country, as they are standardised to give a similar score in different parts of the country, irrespective of local education standards. However, received opinion, the accumulated experience of those who have work across the country and a small amount of research suggests that education standards in Java are generally significantly higher than most places outside Java and that urban areas have higher standards than rural areas. When similar tests to these were done during the PEQIP project a large gap in performance existed between NTT (very low scores) and Yogyakarta and Bali (high scores).
[ HOME - Bahasa Indonesia ]
[ HOME - English Language ]
|
