On problems, and researching problems

Our class discussion on September 6 went reasonably well, although I wish we had a bit more time to go over the problems—of course, I probably lingered on them too much. Meanwhile, Sean has compiled the results of the sticky-note exercise and I will post those results here, so I think we will have a sense of a developing consensus of what the key problems are. I think we might already have five pretty obvious groups, so just about everyone will be in a group that gets to address a problem of interest to all the members.

 

But it appears that the groups will start out as “topic groups,” not “problem groups.” Why topics? Because few, if any, of the sticky notes really described a problem. While I did say in the assignment  that you should frame your problem based on your idea of what a problem is, most of the “problems” were “topics” that weren’t particularly well thought through and which didn’t really reflect a great deal of thought.

 

In particular, they didn’t think of problems. Topics are not problems. “The sky is blue” is a statement, not a problem. Of course, you did better than this—you identified issue areas or topics where there are problems. “Immigration reform” suggests that there is some sort of problem with immigration in the United States. But it doesn’t really explain what the problem is. For some, immigration reform means amnesty for illegal aliens, a guest worker program, and allowing illegal immigrants’ children to attend schools. For others, “immigration reform” means a strict enforcement of existing laws, including deporting anyone working illegally in the United States, stiff penalties against employers of illegal aliens, and so on.

 

In this case, Eugene Bardach, who wrote a great book, A Practical Guide for Policy Analysis, notes that, when defining a problem, “it often—but not always—helps to think in terms of deficits and excess. For instance;”

 

[The following are directly quoted from Bardach, page 2:]

 

How else might we think of problems? An interesting by Gary K. Clabaugh & Edward G. Rozycki http://home.comcast.net/~erozycki/ProbQues.html says that the definition of “what is a problem” is a function of one’s choice to be analytical or an advocate. The line between these two positions, however, is quite blurry. Still, their description of what an analyst does is quite interesting. An analyst, they say, defines a problem as

 

“a. a situation which concerns someone, or

 

“b. a situation which would concern someone if they knew about it.”

 

An advocate, on the other hand, defines a problem as “a problem is a situation which should concern somebody.”[My italics]. See the difference? In the advocate’s case, the goal is to persuade you and me that we should care about global warming, or international terrorism, or the increasing population of deer (really!) or anything else. An analyst works to see if the problem really exists at all.

 

Another place to look for definitions of a problem is a website at Syracuse’s Maxwell School of Government (one of the best of its kind in the nation). The first page asks “what is a social problem?” http://www.maxwell.syr.edu/plegal/TIPS/sp.html The page discusses the various levels at which we can think of the community that is affected by a problem; I strongly suggest that you define your problem in state or national terms because it is often very difficult to do research on local level problems.

 

The other steps are useful as well, although this page does really focus on the local level. The sample list of social problems (http://www.maxwell.syr.edu/plegal/TIPS/sampleprob.html) is good but a bit thin: ignore the school level “problems,” which aren’t really phrased properly (is school morale too high or too low?). Others of these problems are evaluative: “poor academic performance,” and “high dropout rate” for example. Some of the community problems are also properly posed, but most others are not.

 

Bardach’s problem definitions, above, are both analytical and advocacy questions: they say that there are “too many” or “growing faster” which can be both scientific and political questions. In each problem there is, from an analytical perspective, an implicit “therefore…” For example, 'there are too many homeless people in the United States,’  therefore, major strains are put on law enforcement, social services, or the like. Or “the population of school age children is growing” faster than we can build facilities; therefore, schools will be crowded, and steps will have to be taken to address overcrowding, including accelerating the pace of school building, going to year round schools, double-shifting schools, and so on. Each of these outcomes would then be evaluated on their likelihood of coming true.

 

In our class discussion, some of the problem statements made the error, as Bardach descirbes it, of “[defining] the solution into the ‘problem.’” Bardach goes on to give an example: “’There’s too little shelter for homeless families.’ This formulation might inadvertently imply that “more shelter” is the best solution and might inhibit you from thinking about ways to prevent families from becoming homeless in the first place. Try instead: ‘There are too many homeless families.’”

 

A particularly useful worksheet at http://www.maxwell.syr.edu/plegal/TIPS/worksheet2.html will help you organize your evidence of the problem. (It’s also available in Word as http://www.maxwell.syr.edu/plegal/TIPS/worksheet2.doc)  But where the worksheet says “cite at least one source of data” you should be sure that you are citing one source of data (or evidence) for each claim, not for the whole problem.

 

Finding and using quality evidence

Evidence was one of the major shortcomings of the problem exercises. Frankly, in most cases I am not sure that students devoted more than about 1 hour to surfing the web for information, rather than finding information. Taking some cues from the Maxwell pages, quality evidence looks like this:

 

  1. Statistical data, from sources like the Census Bureau, the Bureau of Labor Statistics, and the like. Indeed, you can find links to these excellent sources by going to http://www.econdata.net/ and selecting “Ten Best Sites” from the left menu bar, or just go directly to http://www.econdata.net/content_tenbest.html. The data will not simply fall into your lap—you have to gather, format, and interpret the data, and present it in a way that an educated non-expert citizen can understand.
  2. Articles in peer reviewed scientific and social scientific journals. If you want to know more about what a “peer reviewed” journal is, go to http://www.library.uiuc.edu/alx/peer.htm, http://valinor.ca/peer-review.html, or the Wikipedia discussion of Peer Review (which also describes grant proposals) at http://en.wikipedia.org/wiki/Peer_review . Other articles about what a peer review means can be found by going to Google and typing “what is a peer reviewed journal?” in the search box (without quotation marks.”
  3. Books by academic and technical experts. Books are sort of peer reviewed, but not as closely as some journal articles. The NCSU library catalog has this material, and you can sometimes find books easier by looking at amazon.com and then trying to find the book in our library.
  4. Technical reports written by government or other experts that are not peer reviewed, but that come from quality sources. Such sources include the Government Accountability Office (GAO at http://www.gao.gov/) and Congressional Research Service reports, which are found at the National Council for Science and the Environment at http://www.ncseonline.org/NLE/CRS/). Not all CRS reports are found here. Agencies also publish reports that can be found at the Government Printing Office, http://www.gpoaccess.gov/. Remember, however, that many top-level reports will be written to reflect that administration’s political and ideological preferences—this material may not be “neutral”
  5. Books by journalists and nonexperts. Some are good, others are awful. You have to use your judgment. For example, a book called The Weight Loss Cure They Don't Want You to Know About, by Kevin Trudeau, is very popular. It is also very wrong, and the author is, in fact, a convicted criminal who is allowed to publish books because he has the first amendment right to do so; he is banned from many other business enterprises. See the (itself controversial) discussion at http://en.wikipedia.org/wiki/Kevin_Trudeau)   
  6. “Elite” newspapers such as the Washington Post, New York Times, and Wall Street Journal, Remember, however, that editorials in these papers are not as “neutral” as the regular news stories, and even the news stories are not entirely neutral in one way or the other. Most bias in journalism is not ideological bias: it’s structural bias that causes newspapers and other news outlets to gather the easiest to find news from official sources, press releases, news conferences, etc.

    If you do use these papers, try to track down their sources and use the material the journalists themselves cite—Google and Google scholar (http://scholar.google.com) are particularly well suited for this sort of searching.
  7. Popular magazines, newspapers, and the like. Most people significantly and inappropriately relied on these generally sketchy sources. As you gain some expertise in your problem, you will learn to find the many errors that journalists make—and you will be properly appalled by them.

 

I hope you find this note useful—we will discuss this further in class.