_ [Contents]

Copyright © 2003 jsd

1  Introduction: Types of Ill-Posed Problems

Students often complain about “story problems” (in contrast to problems where you just plug given numbers into given formulas and turn the crank, i.e. “plug-and-chug” problems).

There are lots of reasons why story problems are harder than plug-and-chug problems:

  1. Sometimes the statement of the assignment contains extraneous information, so you have to figure out what to ignore.
  2. Sometimes the statement of the assignment omits some of the crucial information, so you have to look elsewhere to find it.
  3. Sometimes it isn’t obvious what formula(s) to apply.
  4. Sometimes the assignment is ambiguous (also called underspecified, underdetermined, or underconstrained). There are multiple possible answers, depending on what assumptions you make.
  5. Sometimes the assignment is intractible. That is, an answer exists in principle, but there’s no reasonable way of finding it.
  6. Sometimes the assignment is inconsistent (also called overspecified, overdetermined, or overconstrained). No solution exists even in principle, because of requirements that are inconsistent with other requirements, or inconsistent with the laws of nature.

If you are wondering whether you have to do story problems, the answer is yes. Get used to it.

Presumably you are not planning to spend the rest of your life in school. In the real world (i.e. outside of school) you will never see a plug-and-chug problem. It’s 100% story problems, and more often than not, ill-posed story problems.

Suppose you are at work, and the boss or the customers come to you with a question. Do you really think they are going to ask you to plug given numbers into a given formula? The customers aren’t idiots; if they had the numbers and had the formulas they would have turned the crank themselves. They are coming to you because they want you to do something they could not immediately do themselves. They want you to use your judgement. They want you to track down the missing information and figure out what formulas to apply. If you won’t do the job, you’ll be fired and replaced by somebody who will do the job.

An extremely underspecified task is sometimes called “a message to Garcia” – as explained in reference 1.

The most trouble comes from inconsistencies (item #6 on the list above). Presumably you’ve been taught that it’s important to follow instructions. But in the real world you can’t always follow the instructions, because the instructions are wrong.

Dealing with ill-posed problems requires high-level thinking. This is often called critical thinking to distinguish it from lower-level forms of thought. See reference 2 for more about thinking skills and how to sharpen them.

Specific recommendations for how to deal with an ill-posed problem are presented in section 7. But first, let’s discuss some examples.

*   Contents

2  Some Examples

2.1  Packing

I once posed the following question as homework:

"I buy milk in ordinary standard one-gallon plastic jugs. I have a picnic cooler with inside dimensions 12"x12"x12". How many such milk jugs can I fit in the cooler?"

Some of the students were incensed that I would ask such a question without telling them how big a milk jug is. Obviously you can’t answer the question without that information.

I realize that you were not born knowing how big a gallon jug is. The topic has not been discussed in class. It is not covered in the textbook either. But you must not whine about it. Figure it out! FIGURE IT OUT! Your mission, should you decide to accept it, is to figure out how big a milk jug is. Maybe you don’t buy milk in one-gallon jugs. Maybe you don’t buy milk at all. But even so, it wouldn’t kill you to go to the store and measure one as it sits there on the shelf.

Figure it out.
     

If I had asked you explicitly to figure out how big a milk jug is, you would have been able to do it. The task here isn’t really any harder; you just need to know the rules. The rule is that such a question is implicitly asking you to figure out how big a milk jug is.

2.2  Mississippi River

Here’s an example that can be discussed without taking time to dig up additional information (such as the size of a milk jug). It can be solved in a classroom setting ... although it works better in a one-on-one setting, because different students solve this problem at wildly different rates, and it is sub-optimal if one or two students “run away” with the problem while the others are still struggling with it.

The assigment is to estimate how much water flows down the Mississippi river at (say) New Orleans.

Most people, the first time they hear this question, will probably say they have no idea. But that’s a cop-out. Anybody who grew up in the United States probably knows enough facts to be able to figure it out, to an order-of-magnitude approximation, without looking anything up. It requires marshalling quite a few odd bits of information, but it’s quite doable. Practically everybody concedes in retrospect that they should have been able to do it.

So: figure it out.

Now things get really interesting. After you have figured it out, set aside that solution and find another inequivalent solution. (I know of two good solutions; there may be more.)

Many people find it much harder to find the second solution. The second solution is not inherently trickier; the only problem is that the first solution somehow impedes the search for additional solutions. But keep going. Remember the feeling involved. This is a good exercise for out-of-the-box thinking.

Note that the numerical answer is vastly less interesting than the methods of solution. That’s my point: methods can be discussed. Methods can be taught.

2.3  Following Instructions: Closed for Business

Once upon a time I was working with my brother in his law office. It was a Saturday, and he had his kids with him. At about 2:00 his 2-year-old daughter announces "I think it’s high time for lunch."

My brother says, "Yes, sweetie, it’s high time for lunch. Where would you like to go?"

She says, "Eegee’s. They have really good hot dogs."

So we go trundling off to Eegee’s. But when we get there, the parking lot is empty, and there’s a big sign in the door that says "CLOSED. Please come again later."

We were about to drive off, but I said "Wait a minnit. It doesn’t make sense for a joint like this to be closed at 2:00 on a Saturday. Let’s go take a look." Sure enough, the place was open. Six or eight staff were lounging behind the counter, wondering why there were so few customers.

I told the staff "It’s a good thing we didn’t follow the instructions, or you’d have no customers at all. If you turn that sign around, I’ll bet business will pick up considerably."

Over lunch we had a good discussion of the importance of not believing every sign and not following every instruction.

Of course it could have gone the other way: I can imagine walking up to the restaurant and finding it closed. “Of course it’s closed” says the bystander; “can’t you read?”

2.4  Following Instructions: Hand-Held Football

Here’s an example that was a turning point in my life, much more important than milk jugs or hot dogs.

Back when I was a junior in college I designed a hand-held electronic “Auto Race” game, as a consultant for Mattel. Then they came back and asked me to design a follow-on, namely a “Football” game.

They provided a detailed specification. The Football game had to use exactly the same hardware as the Auto Race game (to minimize production costs). The sofware had to fit into the same amount of memory. They specified that it would beep three times when certain game events occured, and beep two times when other game events occurred. They even specified the pitch and the duration of the beeps. And many, many other details.

They forgot to specify that the game was supposed to be fun.

I took their specifications document and put it up on a high shelf, out of sight, and proceeded to design a game. It turns out that you can’t play Football using a steering wheel and a gearshift, so I changed the hardware. And beeping two times or three times isn’t very interesting, so I wrote a subroutine that would play music, and another one that made a tweet-tweet noise to represent the referee’s whistle. The software grew much bigger than the Auto Race software.

In short, I violated virtually every element of the specification.

In retrospect, you might think that each such decision was easy and obvious, but they weren’t easy or obvious at the time. It wasn’t obvious that it was even possible for such a low-power processor to play music. Indeed it wasn’t the least bit obvious that it would be possible to make a decent Football game. If you had asked me in advance whether it was possible to play Football using 27 little red dots and a few pushbuttons, I would have said no, it’s not possible.

After spending months working on this, I was getting quite nervous, because if they didn’t like the game, I would be in serious financial trouble, because they would refuse to pay me. I had, after all, not come anywhere close to fulfilling the contract as written. So with some trepidation I arranged for a meeting at Mattel headquarters, so their engineers and executives could see the demo for the first time.

It worked out. They liked it. After a while I noticed that they weren’t even pretending to evaluate it – they just wanted to play the game. The higher executives were elbowing the lower executives out of the way. They very quickly forgot about all those details in the specification.

I’m a firm believer in giving the customers what they want. But that means giving them what they really want, not necessarily what they say they want.

2.5  Coffee + Cream

Suppose you have some hot coffee and some cold cream. The objective is to make the coffee as cool as possible, within some time limit (say three minutes). The question is, should you add the cream to the coffee immediately, or should you wait until the last moment to add the cream?

Hint: Before mixing, while the coffee is cooling down, the cream is warming up. The relative rates are not well determined by the statement of the problem.

2.6  Snapping the String

Consider the system shown in figure 1. There is a massive object hanging by a thin string, with another string extending below the object.

snap-string
Figure 1: Object + Strings

I know a teaching assistant who used to set up this demo for his students. He said he was going to break the string, and offered to bet them 10 points toward their grade if they could predict whether the string would break above or below the object. Usually there was no shortage of students who confidently predicted that the string would break above the object. What could be more obvious?

The TA would then put some slack in the lower string, and then snap it downwards with a mighty jerk, breaking the lower string against the inertia of the object.

The then re-tied the string, and (since he had a bit of a mean streak) offered the student double-or-nothing if the student could predict what would happen the next time he broke the string. Believe it or not, some students accepted the second bet!

2.7  Yo-Yo

Suppose we have a Yo-Yo set up approximately as shown in figure 2. The outer rim of the Yo-Yo rolls without slipping on the floor. We ask a simple qualitative question: When I pull on the string, does the Yo-Yo move toward me, or away from me?

yo-yo
Figure 2: Which Way Does The Yo-Yo Go?

Huge hint: Principle of Virtual Work.

2.8  South – East – North

You start out at point A. You travel strictly south for one mile. You then make a 90 degree turn and travel strictly east for one mile. You then make another 90 degree turn and travel strictly north for one mile. You find that you have returned to point A.

For simplicity, we approximate the earth’s surface as spherical.

Part 1: Where is point A?

Part 2: How do you know? How sure are you? Can you prove it?

2.9  Sum of Two Rectangles

In figure 3, the black curve represents some raw data. We have lots and lots of data points, with very high precision. We know a priori that the area under the black curve is the sum of two rectangles – a red rectangle and a blue rectangle. All we need to do is a simple “curve fit”, to determine the height, width, and center of the two rectangles. As you can see from the figure, there are two equally good solutions. There are two equally perfect fits. Alas, this leaves us with very considerable uncertainty about the area, width, and center of the blue rectangle.

fit-two-rectangles
Figure 3: Fitting Two Rectangles

3  Regularizers

In figure 3, it may be that both solutions are perfectly reasonable, plausible solutions.

On the other hand, depending on as-yet-unmentioned details, it may be that we have some reason to prefer one solution over the other, as we now discuss.

It may be that we have additional information that we can bring to bear. For example, suppose that we know a priori that the aspect ratio of each rectangle should be close to unity. Then when we do the curve fit, we can construct an objective function that contains not only the usual terms that penalize residuals that stray too far from zero, but also an additional term, a regularizer term that penalizes aspect ratios that stray too far from unity. We find that the left-hand solution in figure 3 does a better job of minimizing this new objective function, much better than the right-hand solution.

Regularizers are a fairly general, fairly powerful way of making an underspecified problem less underspecified. They are a way of constructing an objective function that incorporates the knowledge that we have. This is a huge improvement over an objective function that only knows about the residuals.

If you give too much weight to the regularizer term, it can distort the fit. This is an example of a bias-variance tradeoff. Details are beyond the scope of the present discussion.

4  Approximations

Often an intractible problem can be made tractible by introducing approximations. Document whatever approximations you make.

What separates the professionals from the bush leagues is idea of a controlled approximation.

5  Brittle Laws versus Robust Laws

In the classroom, as in life generally, there will always be tradeoffs between simplicity and accuracy. In an intro-level course, simplicity will be favored. There will never be an exact right way to make the tradeoff ... by which I mean the optimum will be partly determined by taste, and will vary from class to class and even from day to day.

Not only are approximations necessary in order to keep the students from getting brain-blisters ... let’s face it, approximations are necessary to keep the cognitive load on the teachers within reasonable bounds. It is not practical to require teachers to have domain expertise in fluid dynamics and every other field to which physics ideas can be applied.

One might be tempted to just announce "Everything I will ever say is an idealization and an approximation" ... but even that isn’t quite right. Some topics in introductory physics (e.g. conservation of momentum) really are exact, so far as we know, and other parts are close enough for all practical purposes.

This creates a genuine dilemma when formulating story problems. We apply an exact law to a real-world scenario and come up with an inexact conclusion. That’s just plain tricky.

This raises some interesting metaphysical questions about why we think the law is exact, even when all real-world applications are inexact.

This is particularly hard for students who are taking (or have just taken) high-school geometry, where it is emphasized that there is no notion of proximity or continuity when stating theorems. If you mis-state the premises of a theorem even slightly, the conclusion does not hold at all. If you apply a theorem to a case where the premises don’t exactly hold, the conclusion does not hold at all.

As another manifestation of the same idea: If you remember half of your password, you won’t be able to half-way log in.

So why is it that we can approximately apply the laws of physics? We derive laws that apply to massless strings and frictionless pulleys ... but then we apply them to real-world strings and real-world pulleys, and get approximately-correct answers. This is genuinely tricky.

Part of the answer is that often there is a notion of continuity in physical situations. For example, it the thrust is misaligned from the horizontal by some small angle theta, the horizontal component of thrust will be approximately equal to the total thrust. Its magnitude will be off by an amount that is second order in theta.

This leads directly to the notion of controlled approximations as mentioned in section 4. Consider the contrast:

  a) Assume thrust is horizontal.
  b) Assume thrust is nearly horizontal.
  c) Ascertain that being slightly non-horizontal has effects that are second order in small quantities, and therefore unimportant for present purposes.

Another example:

  a) Assume massless strings and frictionless pulleys.
  b) Assume nearly-massless strings and nearly-frictionless pulleys.
  c) Ascertain that if your mass is off by 1% (due to strings) and your force is off by 1% (due to pulleys), your predicted acceleration will be off by about 2%, maybe less, because of the nice continuity properties of the F=ma law.

I really want to emphasize that this sort of continuity cannot be taken for granted. If you randomly mis-type 1% of the characters when writing a long C++ program, your program will not work right, not even approximately.

On the other hand, computers – spreadsheets in particular – are useful for teaching about continuity of physics laws. You can change the input numbers by 1% in each direction, and see how that affects the output numbers.

Some recommendations for teachers:

 1) When writing story problems, don’t demand exact answers.
  1a) Bad: asking about "equal" versus "greater than".
  1b) Better: asking about "approximately equal" versus
   "substantially greater than".
 2) Similarly, don't define things that don't need to be
  defined.
  2a) Bad: Define thrust to be horizontal.
  2b) Better: Say "in this idealized situation, the thrust is
   horizontal."
 3) More generally: be clear, even at the most introductory
  levels, about
  3a) exact statements, versus
  3b) inexact statements.
 4) Be clear, even at the most introductory levels, about
  4a) "brittle" laws, i.e. laws that hold only when their
   premises hold exactly, versus
  4b) "robust" laws, i.e. laws have nice continuity properties,
   so that they can be applied to slightly messy situations.

Remember: You know which laws are brittle and which are robust, but the students weren’t born knowing that ... and this issue isn’t well covered in typical textbooks.

6  Remarks

1) If you want an interesting job, try being a manager in a research lab, “managing” a bunch of people who don’t follow instructions unless they feel like it. It’s like herding cats. These people are very, very successful, because they have good judgement about which instructions to follow and which to ignore.

2) Solving underspecified problems involves taking initiative and taking personal responsibility. Presumably you go to school because you want to learn something. Learning is your responsibility. As the instructor, my job is to help you meet that responsibility. I cannot meet it for you. Possibly some stuff you read in the textbook (and elsewhere) is erroneous. Possibly some of it is irrelevant. Certainly it is incomplete, in the sense that what you learn in school is at most 10% of what you need to function in the real world. Learning is your responsibility; all I can do is help you get started in the right direction.

You cannot escape responsibility for answering a question by finding fault with the question. Life is not a made-for-TV courtroom drama where the objective is to get off on a technicality.

3) I suspect that one of the many reasons why the Harry Potter books are so popular is their portrayal of rule-breaking, with all the negative and positive consequences it brings.

7  How To Answer Ill-Posed Questions

Some recommendations:

  1. If the question as posed omits some information that can be found elsewhere, go find it. Use your initiative.
  2. If the question as posed permits more than one answer, see if one of the answers is “optimal” in some sense. Document what notion of optimality you are assuming.
  3. If the question as posed permits more than one answer, and there’s no clear way to choose among them, try to describe the entire solution set. At the very least, describe a few elements of the solution set, and note that other solutions exist.
  4. If the question as posed is inconsistent or impossible, you should reformulate the question in terms that, in your best judgement, express what was intended ... and then answer the reformulated question.
  5. If you need to use approximate data and/or approximate formulas, document what approximations you are using. Make a controlled approximation. Report an estimate of the uncertainty in the final answer.

8  For Further Reading

1.
Elbert Hubbard, “A Message to Garcia” http://www.birdsnest.com/garcia.htm

2.
John Denker, “Teaching (and Learning) Thinking Skills” ./thinking.htm
[Contents]

Copyright © 2003 jsd

_