Nate Silver, The Signal and the Noise: Why So Many Predictions Fail And Some Don’t” (NY: Penguin Press, 2012), 454 pages.
Nate Silver gained national attention with a forecasting system he developed in 2003 called PECOTA. He used baseball data to forecast the performance of major league baseball players. It was very successful and the attention he got allowed him the time and money to expand his interests into other areas and write us a thoughtful and reflective book on forecasting.
The Signal and the Noise is a book for people who like data and the possibilities for using it in the latest information revolution brought on by cheap and powerful computers. In the Introduction Silver tells a precautionary tale from the first information revolution that came in 1440 with the printing press. Before the printing press knowledge was lost without some way to store it; after the printing press knowledge could be stored but ideas could also be circulated to make arguments to the masses and promote controversies like Martin Luther’s ninety-five theses. The printing and distribution of 300 thousand copies brought centuries of religious warfare.
Silver uses the introduction to make similar contrasts and set up a philosophy of forecasting. Forecasting implies planning with uncertainty that needs prudence, wisdom and industriousness along with a dose of humility. Silver thinks of forecasting as an on-going process of revision where the risk of failure is always present but the possibility of progress makes it worth the trouble.
The book has 13 chapters. The first chapter titled “A Catastrophic Failure of Prediction” describes what went wrong with the predictions for the recent stock market and housing bubbles. Then it is on to six more chapters on prediction for political polls, baseball, weather, earthquakes, economic forecasting and swine flu.
The topics all have a random and unpredictable element for something we would like to predict in advance. Silver gives readers some historical background, some basic theory or science where it’s relevant and then an assessment of the forecasting record: weather forecasting, better, earthquake forecasting, no progress, and so on. Each chapter suggests a common principle or two of success or failure that turns into a general theory and practice of forecasting by the end of the book.
Chapter 8 introduces Bayesian statistical inference in a descriptive form to be applied to six more topics of prediction: gambling, chess, poker, stock prices, greenhouse effects and terrorism. Bayesian inference is a branch of statistics that uses prior probability to make a new probability estimate, the posterior probability. Silver tries to convince readers to think of Bayes as a procedural method that combines new evidence with prior beliefs in a repeated process of revision.
Given the variety of topics from technology and the social and physical sciences readers will prefer some topics over others. I spent more time on the economics chapters. In Chapter 6 Silver does what forecasters hope no one will do: he goes back to check and compare old forecasts. Most are way off but being a labor forecaster I appreciate Silver’s precautions: correlation does not mean causation, explanatory variables change frequently, never throw out data, tell a story using credible economic reasoning.
I especially liked the comments he used of Jan Hatzius, chief economist at Goldman-Sachs. His correct forecast of the 2007 financial collapse resulted from looking at mortgage data and evaluating the size of the leveraged mortgage market and the risk of default from unqualified buyers. At page 196 Silver writes “Hatzius refers to this chain of cause and effect as a ‘story.’ It is a story about the economy – and although it might be a data-driven story, it is one grounded in the real world.”
Chapter 11 takes a close look at the age old question: Can you make money predicting stock prices? Those who believe in markets always answer no, but Silver uses the volumes of stock data to do a variety of fun experiments comparing strategies - buy and hold, manic momentum and a few more – before reaching that conclusion.
The Signal and the Noise is a very readable statistics book with good graphics, thorough documentation and source notes. I would never have predicted publication of a general audience book with so much detail but as I finished reading I decided the volume of data on the Internet promotes a wider interest along with wider access. The growing combination of access and interest make it a timely book that suggests a structure and philosophy for people who want to pursue their own interests and separate the signal from the noise.
Wednesday, July 17, 2013
Monday, July 8, 2013
Inflation and Wages for 2006 to 2012
News of college graduates struggling to pay student loans draws attention to wages. College graduates who cannot find jobs using their college degree skills add to the pool of labor looking for already low wage jobs, potentially lowering wages and buying power even more.
Wages need to keep up with inflation to assure Americans can buy what they produce and keep themselves employed. In practice comparing wages over time requires adjusting wages for inflation with the Consumer Price Index to compare buying power, or real wages. For example, the median wage of Accountants and Auditors was $55,650 in 2006, but to keep up with inflation and have the same buying power in 2012 median wages would have to be $62,188. Instead they were $63,550, a 2.19 percent increase in real wages over the six year period. Data are from the Bureau of Labor Statistics, Occupational Employment Survey, which reports wages for 22 occupational categories and 829 occupations. The self employed are not included here.
Accountants and auditors are the ideal because jobs are up; averaging a little over six thousand new jobs a year with wages up faster than inflation. The outcome for retail salesperson and cashier was the opposite: jobs and real wages declined. The median wage of retail salespersons was $19,760 in 2006, but to keep up with inflation the 2012 median wage needed to be $22,494. Instead it was $21,110, a 6.15 percent decrease in real wages.
The median wage of cashier was $16,810 in 2006, but to keep up with inflation in 2012 median wages needed to be $19,136. Instead they were $18,970, a .87 percent decrease in real wages. Retail salesperson remains America’s biggest occupation with 4.3 million jobs but jobs keep declining; cashier continues to be the second largest occupation with 3.3 million jobs, but also with declining employment.
Retail salesperson and cashier are two of 388 occupations I can find that did not keep up with inflation between 2006 and 2012. These 388 occupations employed 72.5 million in 2006, but only 69.4 million in 2012. Because total employment is down between 2006 and 2012 only a little over 61 million show higher real wages in 2012. I used 2006 as the last full year before the housing bubble and then the recession, but other years could show better or worse results compared to inflation.
The wage data applies to jobs at establishments at the time of the survey and not to people’s wages or income, which are part of another survey. Because jobs turnover, some of the people in low wage occupations in 2006 may well have moved into higher skilled and higher wage occupations by 2012. Therefore, the data allow conclusions about wage inequality but limited conclusions about income inequality.
The results and selected details outlined below suggest two broad statements about wages for the six year period. First, the highest wage occupations in 2006 in upper level management, most of finance and professional occupations have real wage gains by 2012, while the lowest wage occupations in 2006 were the most likely to have lower real wages in 2012. Second, the real wage gains of occupations with wage gains had modest gains, less than one percent a year in a majority of cases. In the United States, working for wages is a tough way to make a living.
Selected Details by Occupational Category-----------------------------
Managerial occupations and financial occupations had the ideal with more jobs and a composite of wages for both occupational categories showing individual occupations mostly keeping up with inflation or with a percent or two increase in real wages. There are exceptions. Managerial occupations had wage gains above inflation for 27 of 36 occupations. General operations managers, food service managers, accommodations managers and construction managers were exceptions. Financial occupations had real wage gains in 24 of 33 occupations. Exceptions included personal financial advisors and real estate appraisers, both down.
Professional occupations did well compared to inflation, although generally a modest 1 to 3 percent increase in real wages for computing and mathematics occupations. Computer programmers are an exception, down slightly. Real wages were down for architects and landscape architects and up for 16 of 18 engineering specialties. Both lawyers and paralegals show a 2.5 to 4 percent decline in real wages. More jobs in legal services and lower real wages suggest the supply is out running demand in law.
Education, training and library occupations had job growth and higher real wages for 46 of 65 occupations in education, training and library. College teaching had higher real wages with some exceptions for professors of education, foreign languages, agricultural and biological sciences. Graduate assistants also had falling real wages. Teaching positions in the public schools had job losses in these years but real wages were up 1 to 3 percent for both elementary and secondary teaching with exceptions in some vocational and special education positions. Librarian lost jobs and buying power to inflation.
Health care occupations had job growth and 45 of 60 occupations with higher real wages. Licensed physicians, pharmacists, therapists, technicians and technologists did well against inflation but there are notable exceptions. Median wages for chiropractors, podiatrists, optometrists, opticians and dentists did not keep up with inflation. Dentists in general practice have high wages, but to keep up with inflation their median wage would need to be $150,421 in 2012. Instead it was $145,240, a -3.44 percent decrease in real wages for the six years. Dental hygienist also lost buying power to inflation.
Registered nurses have 2.6 million jobs, the most in health care and the fifth largest United States occupation. Median wages kept up with inflation, but just barely, rising by .41 percent to $65,470 in 2012. Health care aides and assistants lost buying power even though they are already low paid occupations. For example, nurses aides needed a 2012 wage of $25,248 to keep up with inflation, but it was $24,420, a 3.28 percent decline from 2006.
Home health aides, medical assistants, occupational therapy aides, physical therapy aides, pharmacy aides also lost buying power. The low paid health care occupations have declining real wages and increasing employment suggesting a large increase in the supply of people flooding into low wage jobs because they can not find anything else.
Protective service occupations, food preparation and serving related occupations, personal care and service occupations had more jobs in 2012 than 2006, but 16 of 22 occupations in protective services had lower real wages. Occupations as firefighters, correctional officers and jailors, bailiffs, detectives and security guards along with their supervisory staff had falling real wages. Security guards have over a million jobs, but median wages needed to be$24,508 in 2012 to keep with inflation. Instead they were $23,970, a 2.2 percent decrease in real wages.
Food preparation and serving related occupations had a mix where 10 of 18 occupations with higher real wages along with more jobs. Fast food cooks had the lowest median wage of $15,410 in 2006. Their wages were up more than inflation but they remain the lowest paid job in food services with 2012 median wages of only $18,410. All other categories of restaurant, cafeteria, and even private household cooks have declining real wages. Median wages for waiters, waitresses and combined food preparation and serving workers did better, up more than inflation by 2 to 9 percent, but still only in the low twenties for wages.
Personal care and service occupations had more jobs but lost buying power to inflation in 21 of 33 occupations. Non-farm animal caretakers – typically dog walking services, ushers, lobby attendants, ticket takers, amusement park and recreation workers, barbers and hairdressers, child care workers, personal care aides, recreation workers, and residential advisors had combined 2012 employment of almost 3 million jobs and all with declining real wages.
There were eight more occupational categories with declining employment and generally declining real wages.
Building and grounds cleaning and maintenance occupations had declining real wages in 8 of 10 occupations; janitors and cleaners, maids and housekeeping cleaners, landscaping and grounds maintenance and their supervisory staff among them. Janitors and cleaners have over 2 million jobs, but needed median wages at least $22,687 to keep up with inflation by 2012. Instead they were $22,320, a 1.62 percent decline.
Sales occupations had 17 of 22 occupations with declining real wages including salespersons and cashiers previously mentioned and parts salespersons, advertising, insurance, securities, commodities, and financial services sales agents, real estate brokers and agents and telemarketers.
Office and Administrative support occupations had a few bright spots, but 29 of 55 occupations had declining real wages. Financial support occupations like bookkeeping, accounting, and auditing clerks fared the best with 1 to 2 percent gains in real wages. However, communications operators for directory assistance, long distance, call center and answering services have lower real wages and lower dollar wages. Customer service representatives have over two million jobs, but their 2006 median wage of $28,330 needed to be $32,249 in 2012 to keep up with inflation. Instead it was $30,580, a 5.18 percent decrease in real wages. Receptionists, legal secretaries, desktop publishers, and a selection of shipping, receiving and order clerks had falling real wages between 2006 and 2012.
Farming, fishing, and forestry occupations had fewer jobs and declining real wages in 7 of 14 occupations. Farm workers and crop, nursery and greenhouse laborers were all jobs with declining real wages along with most of logging operations jobs.
Construction occupations did badly with 29 of 59 occupations losing buying to inflation and all the skilled trades like masons, carpenters, drywall installers showing lower real wages and some also lower dollar wages.
Repair and installation occupations had 2 to 6 percent real declines for 33 of 52 individual occupations including automotive and vehicle repair occupations, home appliance repair, telecommunications repair and a few more.
Production occupations had declining employment and 61 of 115 occupations had lower real wages. Team assemblers have 1.2 million jobs, the biggest production occupation. Median wages kept up with inflation, but just barely, rising by .38 percent to $27,640 in 2012. Otherwise assembly and fabrication occupations like electrical and electronic assemblers, engine and machine assemblers had declining real wages from 2006 to 2012.
Other production occupations with declining real wages were in metal and plastic work for jobs as machinists, tool and die makers, machine operators for cutting, grinding, drilling, and buffing; in food processing for jobs as bakers, meat, poultry, and fish cutters and trimmers, slaughterers and meat packers; in apparel and finishing jobs as sewers, upholstery and textile machine operators.
Transportation and moving occupations had declining employment and 28 of 52 occupations with falling real wages. Hand laborers and freight, stock, and material movers have almost 2.4 million jobs, but their 2006 median wage of $21,220 needed to be $24,156 in 2012 to keep up with inflation. Instead it was $23,890, a 1.1 percent decrease in real wages.
Airline pilots and flight attendants, traffic controllers, locomotive engineers, most railroad jobs, and many driving jobs including heavy and tractor trailer drivers, industrial truck and tractor operators and school bus drivers had declining real wages. Pilots, copilots, and flight engineers had median wages of $141,090 in 2006, but to keep up with inflation and have the same buying power in 2012 median wages would have to be $160,200. Instead they were $114,200, a 28.90 percent decrease.
Flight attendants had median wages of $53,780 in 2006, but to keep up with inflation and have the same buying power in 2012 median wages would have to be $61,220. Instead they were $37,240, a 39.17 percent decrease, the biggest decrease I can find among United States occupations.
Wages need to keep up with inflation to assure Americans can buy what they produce and keep themselves employed. In practice comparing wages over time requires adjusting wages for inflation with the Consumer Price Index to compare buying power, or real wages. For example, the median wage of Accountants and Auditors was $55,650 in 2006, but to keep up with inflation and have the same buying power in 2012 median wages would have to be $62,188. Instead they were $63,550, a 2.19 percent increase in real wages over the six year period. Data are from the Bureau of Labor Statistics, Occupational Employment Survey, which reports wages for 22 occupational categories and 829 occupations. The self employed are not included here.
Accountants and auditors are the ideal because jobs are up; averaging a little over six thousand new jobs a year with wages up faster than inflation. The outcome for retail salesperson and cashier was the opposite: jobs and real wages declined. The median wage of retail salespersons was $19,760 in 2006, but to keep up with inflation the 2012 median wage needed to be $22,494. Instead it was $21,110, a 6.15 percent decrease in real wages.
The median wage of cashier was $16,810 in 2006, but to keep up with inflation in 2012 median wages needed to be $19,136. Instead they were $18,970, a .87 percent decrease in real wages. Retail salesperson remains America’s biggest occupation with 4.3 million jobs but jobs keep declining; cashier continues to be the second largest occupation with 3.3 million jobs, but also with declining employment.
Retail salesperson and cashier are two of 388 occupations I can find that did not keep up with inflation between 2006 and 2012. These 388 occupations employed 72.5 million in 2006, but only 69.4 million in 2012. Because total employment is down between 2006 and 2012 only a little over 61 million show higher real wages in 2012. I used 2006 as the last full year before the housing bubble and then the recession, but other years could show better or worse results compared to inflation.
The wage data applies to jobs at establishments at the time of the survey and not to people’s wages or income, which are part of another survey. Because jobs turnover, some of the people in low wage occupations in 2006 may well have moved into higher skilled and higher wage occupations by 2012. Therefore, the data allow conclusions about wage inequality but limited conclusions about income inequality.
The results and selected details outlined below suggest two broad statements about wages for the six year period. First, the highest wage occupations in 2006 in upper level management, most of finance and professional occupations have real wage gains by 2012, while the lowest wage occupations in 2006 were the most likely to have lower real wages in 2012. Second, the real wage gains of occupations with wage gains had modest gains, less than one percent a year in a majority of cases. In the United States, working for wages is a tough way to make a living.
Selected Details by Occupational Category-----------------------------
Managerial occupations and financial occupations had the ideal with more jobs and a composite of wages for both occupational categories showing individual occupations mostly keeping up with inflation or with a percent or two increase in real wages. There are exceptions. Managerial occupations had wage gains above inflation for 27 of 36 occupations. General operations managers, food service managers, accommodations managers and construction managers were exceptions. Financial occupations had real wage gains in 24 of 33 occupations. Exceptions included personal financial advisors and real estate appraisers, both down.
Professional occupations did well compared to inflation, although generally a modest 1 to 3 percent increase in real wages for computing and mathematics occupations. Computer programmers are an exception, down slightly. Real wages were down for architects and landscape architects and up for 16 of 18 engineering specialties. Both lawyers and paralegals show a 2.5 to 4 percent decline in real wages. More jobs in legal services and lower real wages suggest the supply is out running demand in law.
Education, training and library occupations had job growth and higher real wages for 46 of 65 occupations in education, training and library. College teaching had higher real wages with some exceptions for professors of education, foreign languages, agricultural and biological sciences. Graduate assistants also had falling real wages. Teaching positions in the public schools had job losses in these years but real wages were up 1 to 3 percent for both elementary and secondary teaching with exceptions in some vocational and special education positions. Librarian lost jobs and buying power to inflation.
Health care occupations had job growth and 45 of 60 occupations with higher real wages. Licensed physicians, pharmacists, therapists, technicians and technologists did well against inflation but there are notable exceptions. Median wages for chiropractors, podiatrists, optometrists, opticians and dentists did not keep up with inflation. Dentists in general practice have high wages, but to keep up with inflation their median wage would need to be $150,421 in 2012. Instead it was $145,240, a -3.44 percent decrease in real wages for the six years. Dental hygienist also lost buying power to inflation.
Registered nurses have 2.6 million jobs, the most in health care and the fifth largest United States occupation. Median wages kept up with inflation, but just barely, rising by .41 percent to $65,470 in 2012. Health care aides and assistants lost buying power even though they are already low paid occupations. For example, nurses aides needed a 2012 wage of $25,248 to keep up with inflation, but it was $24,420, a 3.28 percent decline from 2006.
Home health aides, medical assistants, occupational therapy aides, physical therapy aides, pharmacy aides also lost buying power. The low paid health care occupations have declining real wages and increasing employment suggesting a large increase in the supply of people flooding into low wage jobs because they can not find anything else.
Protective service occupations, food preparation and serving related occupations, personal care and service occupations had more jobs in 2012 than 2006, but 16 of 22 occupations in protective services had lower real wages. Occupations as firefighters, correctional officers and jailors, bailiffs, detectives and security guards along with their supervisory staff had falling real wages. Security guards have over a million jobs, but median wages needed to be$24,508 in 2012 to keep with inflation. Instead they were $23,970, a 2.2 percent decrease in real wages.
Food preparation and serving related occupations had a mix where 10 of 18 occupations with higher real wages along with more jobs. Fast food cooks had the lowest median wage of $15,410 in 2006. Their wages were up more than inflation but they remain the lowest paid job in food services with 2012 median wages of only $18,410. All other categories of restaurant, cafeteria, and even private household cooks have declining real wages. Median wages for waiters, waitresses and combined food preparation and serving workers did better, up more than inflation by 2 to 9 percent, but still only in the low twenties for wages.
Personal care and service occupations had more jobs but lost buying power to inflation in 21 of 33 occupations. Non-farm animal caretakers – typically dog walking services, ushers, lobby attendants, ticket takers, amusement park and recreation workers, barbers and hairdressers, child care workers, personal care aides, recreation workers, and residential advisors had combined 2012 employment of almost 3 million jobs and all with declining real wages.
There were eight more occupational categories with declining employment and generally declining real wages.
Building and grounds cleaning and maintenance occupations had declining real wages in 8 of 10 occupations; janitors and cleaners, maids and housekeeping cleaners, landscaping and grounds maintenance and their supervisory staff among them. Janitors and cleaners have over 2 million jobs, but needed median wages at least $22,687 to keep up with inflation by 2012. Instead they were $22,320, a 1.62 percent decline.
Sales occupations had 17 of 22 occupations with declining real wages including salespersons and cashiers previously mentioned and parts salespersons, advertising, insurance, securities, commodities, and financial services sales agents, real estate brokers and agents and telemarketers.
Office and Administrative support occupations had a few bright spots, but 29 of 55 occupations had declining real wages. Financial support occupations like bookkeeping, accounting, and auditing clerks fared the best with 1 to 2 percent gains in real wages. However, communications operators for directory assistance, long distance, call center and answering services have lower real wages and lower dollar wages. Customer service representatives have over two million jobs, but their 2006 median wage of $28,330 needed to be $32,249 in 2012 to keep up with inflation. Instead it was $30,580, a 5.18 percent decrease in real wages. Receptionists, legal secretaries, desktop publishers, and a selection of shipping, receiving and order clerks had falling real wages between 2006 and 2012.
Farming, fishing, and forestry occupations had fewer jobs and declining real wages in 7 of 14 occupations. Farm workers and crop, nursery and greenhouse laborers were all jobs with declining real wages along with most of logging operations jobs.
Construction occupations did badly with 29 of 59 occupations losing buying to inflation and all the skilled trades like masons, carpenters, drywall installers showing lower real wages and some also lower dollar wages.
Repair and installation occupations had 2 to 6 percent real declines for 33 of 52 individual occupations including automotive and vehicle repair occupations, home appliance repair, telecommunications repair and a few more.
Production occupations had declining employment and 61 of 115 occupations had lower real wages. Team assemblers have 1.2 million jobs, the biggest production occupation. Median wages kept up with inflation, but just barely, rising by .38 percent to $27,640 in 2012. Otherwise assembly and fabrication occupations like electrical and electronic assemblers, engine and machine assemblers had declining real wages from 2006 to 2012.
Other production occupations with declining real wages were in metal and plastic work for jobs as machinists, tool and die makers, machine operators for cutting, grinding, drilling, and buffing; in food processing for jobs as bakers, meat, poultry, and fish cutters and trimmers, slaughterers and meat packers; in apparel and finishing jobs as sewers, upholstery and textile machine operators.
Transportation and moving occupations had declining employment and 28 of 52 occupations with falling real wages. Hand laborers and freight, stock, and material movers have almost 2.4 million jobs, but their 2006 median wage of $21,220 needed to be $24,156 in 2012 to keep up with inflation. Instead it was $23,890, a 1.1 percent decrease in real wages.
Airline pilots and flight attendants, traffic controllers, locomotive engineers, most railroad jobs, and many driving jobs including heavy and tractor trailer drivers, industrial truck and tractor operators and school bus drivers had declining real wages. Pilots, copilots, and flight engineers had median wages of $141,090 in 2006, but to keep up with inflation and have the same buying power in 2012 median wages would have to be $160,200. Instead they were $114,200, a 28.90 percent decrease.
Flight attendants had median wages of $53,780 in 2006, but to keep up with inflation and have the same buying power in 2012 median wages would have to be $61,220. Instead they were $37,240, a 39.17 percent decrease, the biggest decrease I can find among United States occupations.
Subscribe to:
Posts (Atom)
