We refuse to become more efficient

Interesting story in the SMH today about "secret" cuts to education in NSW.

What interests me about the story is that it talks about cuts, but also talks about the education budget blowing out to $13.6 billion by next year if savings are not made.

So what they are saying is that the budget will still go up - just not by as much as it would have before some savings were made.

Apparently a failure to increase a budget by a large amount is now a "cut".

Here's my favourite paragraph:

One principal who chose not to volunteer for the trial said: ''The notion of devolving responsibility to principals is a deceit because in reality it is about making principals do much more with much less.''

Welcome to the real world, buddy. It's all about doing things more efficiently and effectively - that's what "doing more with less" is all about. It's about exploring better ways to do things - improving productivity and cutting wasted effort.

I'd start by sacking that useless tool. Any manager that doesn't see their role as including continuous improvement and the constant search for efficiencies is not a manager in my book. They're a blood sucking leech with a bottomless appetite for other people's money.

NAPLAN vs school funding

I have lost count of the number of times I've been told that the key to improving the education of the "disadvantaged" is more money. I wonder about that - can more money convince feckless parents to read to their kids before bedtime? Can more money convince useless parents to force their kids to do their homework and do some study every night?

Now that I have more data, I've updated my graphs on school funding vs NAPLAN scores in grades 3 and 5. The R-squared results are low 0.32 to 0.35, so there's stuff all correlation between NAPLAN results in those years and how much money is stuffed into a school.

Notice that the line slopes down though - the less funding a school gets, the better its NAPLAN score tends to be. That shows that schools with low NAPLAN numbers are getting more money - the big question is whether they are putting it to good use. What's the point of giving them more money if all they do is piss it up against the wall? By that, I really mean the parents pissing it up against the wall. A school and its teachers can only do so much. Parents have to do their bit too. A good school will always have a tough time producing results if the parents don't provide any support and direction to their kids.

Just how are things going in Wilcannia?

See that red oval out on its own? That's the NAPLAN results for Wilcannia, a state school with 109 students - 96% Aboriginal. It's out in the middle of nowhere.

The NAPLAN results suggest that by year 9, the kids in that school are at about the same level as city kids are in year 3. And we're talking about city kids from the arse end of town. They're well behind the city kids in year 3 that go to state schools in posh suburbs.

Six years behind. Fucking helski.

MySchool - just how far behind can a school fall?

I've extracted some more data from the Chatswood and Fairfield areas regarding high schools, and applied it to my favourite graph - the one showing how the NAPLAN scores of schools improve over time. The problem with this data is that it's not continuous - you have kids going from primary school to high school, so there is a break in the data between years 5 and 7. On top of that, NAPLAN testing hasn't been going long enough to follow the same set of kids from grade 3 all the way to grade 9. The results for years 3 and 5 are from a different set of kids from years 7 and 9.

No matter - we just have to work with what we have.

The bottom axis is the ICSEA score - a measure of wealth. Poor areas to the left, rich areas to the right. The blue data points are the NAPLAN scores in year 3, the brown ones are the scores in year 5 etc etc. You can see how the NAPLAN results climb over time.

What surprised me is that the rich schools pull away in high school - particularly the state schools. There are a lot of very expensive independent schools on the North Shore, and the elite state schools leave them in the shade. More on that another time.

Have a look at the results of the school in the red circle. What you're looking at there is the NAPLAN score for a Fairfield school for its year 9 students. The long red line shows where it sits in relation to state schools in a rich area. By my reckoning, kids in year 9 at that poor school are at about the same level as kids in year 4 at the schools in the rich area. That's the difference between a state school in a good area and one in a bad area.

More MySchool - just to annoy Cav

The theme for this evening is - chuck around a few hypotheses and see what sticks. I was always useless at defining a null hypothesis way back in my school days, so I'm not even going to try to do that now. I always buggered it up. I've spent the afternoon pulling down more stats and combining them to see what falls out.

Big caveat up front - this covers a limited number of primary schools in NSW. This is interesting stuff, but I'd need access to lots more data before staking my life on the results.

First idea - is there a relationship between the ratio of teachers to students and the amount of funding per student?

Answer seems to be "yes". I would expect that to be true since teacher's salaries are the biggest cost for schools. The R-squared is 0.688, making it a good fit. Part of the remaining variation could be explained by variations in mix of teacher salaries across schools - that is, one school might have 10 newbie teachers on $50k, and another might have 10 old farts on $80k. If they have the same number of students, the teacher/student ratio will be the same, but the second school will require funding 60% higher.

Just remember this - with state schools, over 80% of the bills are paid centrally. The schools never see much money - head office pays the staff, pays the maintenance bills, buys the computers etc etc. The numbers that have been released by ACARA are the result of the head office bureaucrats at the DET taking about $9 billion worth of expenditure and then trying to work out how to divvy it up between a few thousand schools.

I then split the stats for Catholic and state schools. Both show a very strong relationship between funding per head and student/teacher ratios.

However, we have to be careful when interpreting this - which is driving what? Remember, funding for schools is producer driven, not consumer driven. It's a chicken and egg question - does a school have a lower student/teacher ratio because it gets more funding, or does it get more funding because historically it had a low student/teacher ratio? Budget setting in the public sector is a strange beast - the amount of money you get can be based on the most illogical reasoning's at times.

Moving on - is there a relationship between the size of the school (number of students) and the NAPLAN scores? Is a small school better than a big school?

The graph above suggests that it makes no difference whatsoever. Big or small - doesn't matter. It might make a difference to the range of subjects you can study, but for NAPLAN results - zip.

However, I have coloured the data points to differentiate rich areas from poor areas. Blue are wealthy, brown are poor. There's a pretty clear split - the wealth and educational background of the parents counts an awful lot more.

Why do bureaucrats like big schools? Because it makes their life easier when dealing with a small number of big entities rather than a large number of small entities. They'll claim of course that bigness brings the benefits of scale, but the only things that I have ever seen scale well in a bureaucracy are stupidity, ineptness and bloodymindedness.

I then took the NAPLAN scores for a bunch of schools and looked at the performance of their grade 3 cohorts in 2008 and those same cohorts in grade 5 in 2010. I wanted to see how their scores improved over that period across a range of schools. Chatswood is a wealthy area - the average NAPLAN total went up 425 points. Why did some schools jump further than others? Has money anything to do with it?

Just to show you what I'm doing, here are 5 schools from Chatswood. The rhomboid at the bottom is their grade 3, 2008 NAPLAN score. The square up above is the NAPLAN score for that same class when it was tested in grade 5 in 2010. Along the bottom axis is funding per student. Did having more money produce a bigger jump in grades? It did for one school, but not the rest.

Here's the same schools, but with the data presented a different way. I've shown the student/teacher ratio on the bottom axis this time. The further a school is to the left, the better the student/teacher ratio (ie, supposedly smaller classes). The school with the biggest jump (in purple) had a low ratio (small classes) - however, the 2nd biggest jump was the blue school, which had a higher ratio. The rest were all the same - didn't matter if they had smaller classes or bigger classes, the improvement was the same.

The green school and the purple school both started at low NAPLAN scores, and both had small class sizes. The purple school leapfrogged ahead of two of the others with larger class sizes, but the green school didn't. Small class sizes might be beneficial - or they might not. Anyway - you need to crunch lots more than five schools to get an answer.

Which is what I did. I looked at how much the NAPLAN score went up for 70 schools, and then compared that to the student/teacher ratio. According to the R-squared, there is no relationship whatsoever. It's a paltry 0.03.

If you look at the percentage increase in NAPLAN scores vs the student/teacher ratio, you get a slightly better R-squared of 0.15, but this still means there is next to no relationship there at all.

I then split out the Catholics and the state schools, just to see if there was any difference. With the Micks, it's next to nothing.

State schools gets a slightly improved relationship, but an R-squared of 0.23 still means that there is two thirds of bugger all correlation.

Here's where things could get interesting. The blue dots are the grade 3 NAPLAN scores from 2008 for schools in a rich area and a poor area. Along the bottom axis is the ICSEA score - the measure of family wealth. There's a pretty clear pattern - schools in poor areas get much lower NAPLAN scores than schools in wealthy areas. The difference between the worst performing school and the best is 682 points - 1783 vs 2465. The schools in wealthy areas start out with an advantage of up to 38%. The difference between the averages across the two groups is about 14%.

The brown data points are the grade 5 NAPLAN scores two years later for those same schools. The same pattern appears - the rich schools stay ahead, but the average gap shrinks to 9%. Shockingly though, the lead of the best school over the worst stays at around 36%.

I've circled the lowest performing school from 2008.

Have a look at where its students get to by grade 5. Compare it to the schools in rich areas circled in green on the right. That green circle is the rich school scores from grade 3. For the grade 5 kids in the worst school in the poor area, when they hit grade 5, they are performing at the level of the worst performing rich area schools - from grade 3! Not only are they two years behind, but they're even at the back of that pack!

PS - remember, these are all state schools and low fee Catholic schools. What you're mainly seeing here is the difference in performance between a set of state schools in a rich area and a set of state schools in a poor area. In case you are wondering about funding, the poor state schools got an average of $9,498 each and the rich state schools got an average of $8,480 each - about a thousand bucks less (or the poor schools got 12% more). The best funded poor school got $10,894 - the lowest funded rich school got $7,057 (or the best funded poor school got 50% more than the worst funded rich school).

That rich school with the least money - only two schools out of this entire group got higher NAPLAN scores. The school with the highest NAPLAN score (in a rich area naturally) actually got the second lowest funding allocation out of all of them.

Anyway, that's just a taste of the sort of number crunching that can be done with these stats. ACARA, being a bunch of bastards, have made it as difficult as humanly possible to extract the numbers to do this sort of analysis, so it's both time consuming and shallow.

More posts on MySchool:

Number 6

Number 5

Number 4

Number 3

Number 2

Number 1

Damn, stuffed up teh numbering.

Number 0.

MySchool - why is it so?

Arguments regarding the amount of funding that different schools are getting are likely to rage on for the next 10 years. One statement that I read today was along the lines of, "State schools get less money per student because they are larger, and thus more cost effective". I thought I'd put that to the test by looking at the data from the Chatswood area - you make up your own mind.

First graph - size of school vs funding per student. The R-squared is 0.4 - not that crash hot. However, it does appear that larger schools get less funding per head - there is some relationship there. Just remember that correlation does not mean causation.

I decided to just look at State Schools alone - the R-squared drops dramatically to 0.28 - which means almost no correlation. This needs to be tested against a bigger data set, but so far, that idea is not looking good - at least in the state system. It could turn out to be an urban myth. Or more data might prove it.

But do we just care about cost, or is performance more important? I therefore decided to look at the NAPLAN scores for grade 3 vs the size of the school. When Catholic and State Schools are mixed there is almost no correlation whatsoever.

However, when you split them out, look at the difference. For Catholic schools, it appears that smaller schools deliver better NAPLAN results - the R-squared is 0.45, which is sort of reasonable but definitely not great. There is a weak correlation there.

But look at the state schools! The bigger the school, the better the results - and the R-squared is 0.7, which is pretty damned good.

Just remember one thing - this is looking at bugger all data. It's late, and I'm going to bed. I will crunch more numbers tomorrow and over the weekend.

1735099 - what's your opinion on this?

Sorry, Cav

I didn't get any good photos of bikes today, so it's back to crunching numbers from MySchool again. Yes, I must be weird - I find doing a spot of number crunching to be quite soothing and engaging.

Tonight's exercise was to pick another area of Sydney and to bring in another 15 school's worth of data. I chose the Auburn-Granville area, which is probably best known as being the home of a bloody big mosque. Most of the suburbs in the area are on the poor side, so pulling them in just added another slab of green dots to the graph above.

However, one dot appeared up the top - I had a look into it, and found that I had included the state school at Newington. This is one of those new suburbs that popped up after the Olympics on the old Naval armoury near Homebush. It's a very nice, middle class area. Brand new, very shiny. It's no surprise then that it sticks out like a sore thumb compared to Auburn, and has results on par with rich areas like the lower north shore and the inner west.

I've stuck in three purple markers though - these are middle income suburbs, but their results are crap. I went back and had a look at the data - these places are probably well known to most Sydneysiders as the home of drive-by shootings and Muslim gangsters. So the residents there are doing OK economically, but living in a crime infested neighbourhood in the midst of lots of gangsters is not that great for the school results. As 1735099 commented, values and community has a lot to do with it.

So next time someone starts raving that public education needs wads more money, ask them if more money will buy a better set of values. If not, tell them to keep their filthy hands out of your pockets.

MySchool - education bang for your buck

OK, just to keep Cav sane, this is the last post with numbers and graphs and things for tonight. Normal bike bum service will resume tomorrow.

I have made up a new index - the number of funding dollars per student required to produce one NAPLAN point.

That is, if a school gets $8,000 in funding per student, and the NAPLAN score is 2,000, then it takes $4 to produce each NAPLAN point. The lower the number, the more effective a school is at turning our money into education (or at least the ability to do NAPLAN tests - the two are not necessarily the same). But just go with me for a minute.....

I then decided to compare the "BOAB index of educational efficiency" with the ICSEA score for each school. The ICSEA score tells you how wealthy the area is. Let's see how things went....

For starters, there's a pretty wide spread in scores, and the trend line gives us an R-squared of 0.55. Here's how I interpret this graph - the poorer the area, the more money it takes to provide an education. If you compare a rich area with a poor area, it can take nearly double the amount of money to provide the same outcome in the poor area.

Here's the same graph, with the axes set to zero. I don't want you thinking that I'm manipulating things by playing with the axes.

I then separated out the Catholic schools. The R-squared here is 0.0398 - in other words, there's bugger all correlation between the two variables. In Catholic-land, the wealth of an area has almost no impact on how much it costs the Micks to bash knowledge into poor kids and rich kids.

With state schools, it's an entirely different picture. The R-squared here is 0.69 - pretty bloody good in my books. You can also see a clear split between two sets of schools - an upper shoal and a lower shoal.

I've then colour coded the two groups. The red dots are all state schools in rich areas; the green dots are state schools in a poor area. Wow - the difference really stands out.

I've then broken the numbers out for the state schools and divided them into rich and poor zones. The R-squared for the poor group is 0.088 - it's rooted. However, a 5 order polynomial gets up to 0.6.

However, for the rich group, the R-squared is 0.59. By stuffing around with a 5 order polynomial, I got that up to 0.77.

But what does all this tell us?

When it comes to State Schools, it's clearly a two speed system (although I'll have to delve into some other areas that are middle income to see how they fare). There is a massive divide in performance and cost between the state schools in rich areas and the state schools in poor areas. And it has to be remembered that all that extra money in the poor areas isn't buying results - the state schools in rich areas clearly outperform the state schools in poor areas.

I've taken that graph again and coloured in the Catholic schools in orange. A strange colour for Catholics, I know. However, what it shows is that there isn't the same divide as you get in the state system.

What cracks me up about this is that the haters of non-state education love to say that private education creates a gulf between rich and poor. Well, perhaps it does - but the state system is no better. If anything, it might be worse. If you want to prove me wrong, go away and crunch the numbers and see what you can come up with. I went into this exercise with an open mind - I just wanted to throw some numbers around to see what came up. I have to say that I'm quite shocked at the results.

MySchool - and now we examine Chatswood

This is the fourth in a quick series based on the MySchool website. If you've just arrived, scroll down a few posts and start at the beginning. This post is based on Chatswood, a wealthy area on Sydney's north shore. It has been so effectively colonised by successful Chinese immigrants from places like Hong Kong that a former staff member of mine (who was born and raised in HK) called it "Chatswoo", as in "it is full of people called Woo". The Chinese are notoriously pushy when it comes to education. Chatswood Public for instance has 83% of its students coming from LBOTE (Language Background Other Than English) - but it has the 2nd highest NAPLAN score out of all the schools in this sample. Just goes to show that a LBOTE background doesn't necessarily mean a lousy school performance.

The state school and Catholic ICSEA scores for 17 schools in the area are equal on 1189. By comparison, that's about 80 points higher than where I live (the higher the score, the wealthier the area).

The income quartiles paint a very nice picture of the area - there is almost no one in the Chatswood area in the lowest income quartile. The top quartile is packed for both state and Catholic schools.

The amount of funding that the schools get per head doesn't vary that dramatically in this area. Even the Independent school doesn't charge outrageous fees.

The ICSEA scores are also clustered together pretty closely - the scale on the left makes it appear that there is quite a difference, but as I've said before, in a wealthy area like this, the difference in scores is like comparing a family with a 5 series BMW to a 7 series BMW - and then comparing them to the people out in Fairfield in the 3rd hand Corolla.

There is a reasonable spread of NAPLAN results too, with the Catholics doing well up the top. However, all these scores are excellent. Looking at this table is like looking at the Gold, Silver and Bronze winners at the Olympics - the person that took away bronze is still an awesome athlete.

The above graph shows the total NAPLAN scores for Grade 3 compared to school funding per head. Again, this is pretty arbitrary for this area - those scores are all very good, and compared to a poor area like Fairfield, the amount of funding per head is low. In other words, all these schools provide good results at a reasonable cost.

This is where things get funky - this graph shows the results from three areas - a low income area and two high income areas. For this graph, I've increased the size of the markers for the Chatswood schools. What sticks out to start with is that all the Chatswood area schools on the top side of the NAPLAN scores. All of them. Same goes for the Five Dock schools - every one of them is on the top side of the line. With Fairfield, every state school is below that line - only the Catholic schools from that area are performing on par with the state schools and Catholic schools from the richer areas.

The second thing that struck me is that in terms of value for money (or scores for bucks), almost all the Catholic and state schools in the Chatswood area are good value. Most are in the top left hand quadrant - the "stars" quadrant. The four that don't make it are reasonably close to the line.

Here's the big challenge for policy makers - the problem schools are those below the line. In other words, in our sample, every single school in the poor area. They're either low cost and poor performance or high cost and poor performance. How do you improve their performance?

There are those that repeat endlessly that the problem is a lack of resources - that state schools need more resources. Well, I think I have just shown that not all state schools need more resources - those in rich areas are doing fantastically, and they're doing it with the same or less money than the state schools in poor areas.

Let's take the schools in the "sink" quadrant - low cost, low performance. Let's assume we throw more money at them - they can move in two directions. The preferred direction is to move towards "A", which is into the more expensive, but high performance quadrant. The worst possible result is for their results to flatline, but their costs to increase dramatically (option "B"). It would be interesting to look at the finances and results of these schools over time. For instance, were those schools in the bottom right hand quadrant in the bottom left hand quadrant a few years ago? Have they been showered with money, but done nothing useful with it? (Makes you wonder what impact all that BER spending is going to have).

What gets me about the above graph is that we are more likely to follow option "B" rather than option "A". Option A is not about throwing more money at the problem - it is about qualitative improvements in the school. It is about turning around the culture of these schools - something that's probably dreadfully difficult. It really is so much easier to just take more money from the taxpayer and talk loudly about throwing it their way, rather than doing the hard yards of micro reform.

The best possible result of course is to wind back the spending on the expensive under-performers whilst improving their results. Now that would certainly be a challenge - but this is the sort of thing that happens in business on a regular basis. Because if not, the business goes bankrupt. If you're a business and you're in that corner, basically you're fucked. It's time to turn around or liquidate.

Another option is to just accept that perhaps there isn't much that the schools can do for these kids, regardless of how much money we throw at them. What counts more highly is the background and culture of their families, and how much support and encouragement they give their kids. In that case, we do two things - we stop dreaming that all kids can be super stars, and we do our damnedest to identify the smart and capable kids in those areas and we yank them out as early as possible and send them to a good school in a good area, where they are surrounded by kids that support academic achievement. Or we give them a top-up voucher to send them to the local Catholic school if they are too poor to afford the extra fees.

More on MySchool, school income and NAPLAN

Everything in this post relates to primary schools only. I put up two more posts on this subject yesterday - read them first if you are new to this.

The work in this post was suggested by 1735099, who actually teaches, so I hope this adds some value.

What I did yesterday was to crunch the numbers on two different areas - a high wealth area and a low wealth area (Five Dock vs Fairfield). One is full of old migrants who have done well for themselves, and the other is full of new migrants who are struggling to get off the ground.

The idea of this post is to see whether there is a connection between the amount of money that a school gets and the NAPLAN scores the kids get.

In order to do that, I invented the BOAB Naplan index - essentially I took all five of the NAPLAN results for grade 3 and added them up. It's not an average - it's a total. I am the first to admit that it is not terribly scientific, but it's the best that I can be bothered coming up with on a Sunday afternoon.

Here's the results from the high wealth area - it's a bit of a mixed bag. The best NAPLAN scores in our area are delivered by two state schools. I tried adding a trend line - forget it. By the way - the red markers are state schools, and the blue ones are Catholic schools.

I then did the same for the low wealth area. Out west, the Catholic schools perform better (or since this is looking at Grade 3 results, they attract a better class of student). The Catholic schools also deliver a good result at lower cost. The green marker is an Independent school.

Now we apply the good old BCG matrix idea to the graphs. This is all arbitrary by the way - I adjusted the minimum points on both axis, and then drew four quadrants over the data. It's quick and dirty, but it provides an interesting way to think about how schools are performing. As taxpayers (and school fee payers), we want to get the most bang for our buck. We get that in the top left hand corner - schools up there are stars. They get good results at low cost. Schools in the bottom right hand corner are dogs - expensive, low performers. I'm going to call those in the top right corner the "prestige" schools, as they provide good results at higher cost. I'll let you lot think of a name for the bottom left corner - low cost, low results.

Where this gets interesting is when we apply it to the low wealth area. We suddenly find that the Catholics do well in the "star" category - low cost, good results. The state schools do well at getting reasonable results, but at higher cost.

The fun really begins when you compare both types of schools across both areas. Sorry if the graphs get a bit hard to read from this point. In the above graph, the red pointy hats at state schools in low wealth areas. The black hats are Catholic schools in low wealth areas. The red rhomboids are state schools in high wealth areas, and the blue rhomboids are Catholic schools in high wealth areas.

Here, maybe a few lines on the graph will help. What do you see?

Here's what I see. Look at the two state schools above that I've circled. The red triangle is the top performing state school in the low wealth area. The red rhomboid above it is the lowest performing state in the high wealth area. When it comes to state schools, the split is blindingly obvious - if you live in a good area, you get good NAPLAN results, regardless of school income per head.

However, the Catholic schools do well across both areas. Amazingly, the top NAPLAN scores come from two Catholic schools in the poor area.

Also, when you compare across both areas, the relatively expensive state schools in the high wealth area (expensive compared to the local Catholic schools) suddenly look like absolute bargains compared to the amount of money being poured in to state schools in the poor areas.

Here's those two top performing Catholic schools.

All this has to be taken with a pinch of salt, as we are looking at one made up NAPLAN index across two areas, and only for grade 3.

That said, I think it conforms a lot of folk wisdom.

In poor areas, the Catholic schools are attracting the "better" kids (for want of a better term).

In a well off area, it probably doesn't matter what type of school you go to.

My question is this - we're pouring money into state schools in poor areas. Are we getting results commensurate with the effort? I know that for many people, their first reaction will be, "We should put more money into this area". But is that the right reaction? Can you make a silk purse out of a sow's ear?