liz writes stuff down
28Mar/101

A very MIT signals problem

I've always found signals and systems interesting, as it is one of the most power tools out there. Signals and systems can be used to describe many different problems because it is simply an abstraction which describes a physical, mathematical, or computational system by the way it transforms an input signal into an output signal. It's often studied by electrical engineers because it has many direct applications to signal processing and communication systems, but there are lots of applications in other fields.

The signals and systems intro class at MIT, 6.003, is one of the most dreaded and disliked Course 6 classes. The class used to be required for all Course 6-ers, both EE and CS majors, but now that the EE/CS department has switched to a new curriculum, it's only required for double E's. It's a bit unclear where I fall in the Course 6 spectrum, but most of my friends think I'm crazy for having enjoyed 003.

I took 003 in Fall 2009 with Professor Denny Freeman. His approach deviated from the usual approach to the class by

  1. reordering the topics so that Laplace transforms were taught before Fourier transforms and
  2. introducing the concept of "Engineering Design Problems."

The "Engineering Design Problems," or EDPs for short, aimed to show 003 students some tangible applications of the material, and they were open-ended questions which typically required some amount of programming. For most people, these problems made them a bit more excited about signals and systems, but for me, this was all about getting excited about writing little pieces of code.

The most "MIT" EDP was assigned near the end of term:

The following images have been blurred. Figure out a way to sharpen each image to identify the following buildings:

Blurred Buildings

You might be able to guess what some of those buildings are without even seeing the larger images the course staff included in the assignment. (b1 certainly is the most distinctive.)

Of course, there are many ways to blur an image, but looking at this from the simplest 6.003 perspective, it's most likely that either the rows or columns were blurred by a system with a single pole. After all, Denny Freeman is a big fan of Occam's Razor. Running with this assumption, such a system would have a system function with the following form:

H_{blur}(z)=\frac{1-p}{1-pz^{-1}},

where p is the system's only pole. This system would be stable if |p|<1. More importantly, if this system were a low-pass filter, i.e. if 0<p<1, it would blur the image.

You can deblur a system blurred by a single pole by applying a system with a single zero:

H_{deblur}(z)=\frac{1-pz^{-1}}{1-p},

where p is this system's only zero and has the same value as the pole in H_{blur}(z).

Since we will want to write code to deblur the image, we will want to get the difference equation corresponding to H_{deblur}(z) to apply to the rows or columns of the blurred images. The corresponding difference equation is:

y_{deblur}[n]=\frac{1}{1-p}(x[n]-px[n-1]).

Now that we may have figured out what's going on generally, let's look closely at image a1:

Blurred Building a1

The blurring in image a1 looks to be primarly horizontal, which means rows of the image's pixels would have been passed through the low-pass filter. To deblur this image, we should try to pass rows of pixels through the deblurring different equation, y_{deblur}[n]. The rows were processed either casually or anti-causally, i.e. left to right or right to left, respectively.

At this point, we really just have to dive into writing some code. The first deblurring code I wrote was a Python script to deblur a blurred_image from left to right with a pole p and save it to deblurred_image:

import Image
import os

def deblur_left2right(blurred_image, deblurred_image, p):
    original = Image.open(blurred_image)
    new_image = Image.new('L',[original.size[0],original.size[1]],0)
    original_pixels = original.load()
    new_pixels = new_image.load()

    for j in range(original.size[1]):
        new_pixels[0,j] = original_pixels[0,j]
        for i in range(original.size[0])[1:]:
            new_pixels[i,j] = (original_pixels[i,j]-p*original_pixels[i-1,j])/(1-p)

    new_image.save(deblurred_image)

After playing with different values for p, it was apparent that images a1 and a2 were blurred from left to right with a pole at 0.985, so deblurring them with a system with a zero at 0.985 returned the original images. Here is the unblurred version of a1:

Unblurred Building a1

As you can see, a1 is building 68.

The buildings in the second row, b1 and b2, had their columns blurred from bottom to top with a pole at 0.985, and the third row, c1 and c2, had their columns blurred from top to bottom with a pole at 0.985. You can change the for loops in the Python script to deblur in other directions. I encourage you to also see what happens when you change the value of p.

Even with "cute" EDPs like this one, 003 last fall was still all about grungy math; signals and systems often are. However, students who hadn't decided that the class would be too terrible before even stepping into 34-101 for the first lecture seemed to enjoy playing with some of the more tangible applications of signals and systems and got a lot out of the class. Hopefully, more people can come to appreciate this class in its own right, and maybe fewer people will shy away from being an EE because they fear 003.

14Mar/101

A meeting of mindsets

This spring, I finally had time in the right places in my schedule to incorporate teaching into my semester. I'm a Learning Assistant (LA) for 6.042: Mathematics for Computer Science, which offers an introduction to discrete mathematics oriented towards computer science and engineering. The class is taught in a very interactive manner: each hour and a half class session is split into a roughly half hour lecture and a roughly hour long team problem solving session. Additionally, students come from a variety of backgrounds: for many this is their first Course 6 class, whereas some students are Course 6 MEng's; some students have taken many theoretical math classes, while others have only taken one semester of calculus.

As an LA, I work on writing the problems for the class, prepare other materials for the class, coach the students during the class problem solving sessions, and hold office hours. Basically, the roles I've been personally assigned as an LA makes me a TA who doesn't have to deal with grading.

This semester's 6.042 students took their second miniquiz two Wednesdays ago, and their quiz grades were finalized this last Wednesday. The biweekly miniquizzes in 6.042 aim to test our students' understanding of the material covered in lectures, class problems, online tutor problems, and problem sets from the previous two weeks. Theoretically, students don't need to put in a lot of studying if they've been keeping up with those four areas of the course, but of course, an individual student may need more or less help. The miniquizzes typically average around 15 points out of 20, with a standard deviation of about 4 points.

Miniquiz two this semester turned out a bit differently. The histogram for total miniquiz score (I am using an image compiled by another TA, so to clarify, score values are on the x-axis and the number of occurrences is on the y-axis):

Overall Miniquiz Scores

Key statistics for miniquiz 2 grades:

  • 83 students "took" the quiz. It is important to note that 82 students actually took the quiz, and that the score of 0 is for a student who punted the quiz. (We'll be ignoring the student who punted in later statistics.)
  • The mean was 12.50 points.
  • The median was 12.50 points. (No skewness!)
  • The standard deviation was 3.93 points.

Based solely on the above histogram, the quiz scores seem to follow a fairly reasonable bell curve, just with a lower average than expected. From the perspective of the course staff, that would normally just mean we made the miniquiz a bit harder than we'd have liked, but the results of this quiz were more complicated than that because of the scores received on the last problem. The last problem was:

Let [\mathbb{N} \rightarrow \{1, 2, 3\}] be the set of infinite sequences containing only the numbers 1, 2, and 3. For example, some sequences of this kind are:

  • (1, 1, 1, 1...),
  • (2, 2, 2, 2...), and
  • (3, 2, 1, 3...).

Prove that [\mathbb{N} \rightarrow \{1, 2, 3\}] is uncountable.

Hint: One approach is to define a surjective function from [\mathbb{N} \rightarrow \{1, 2, 3\}] to the power set \mathcal{P} (\mathbb{N}).

Having seen proofs for the last six years, the mathematician part of me was not particularly concerned with the difficulty of this problem, and I quickly came up with the following solution (which did not use the hint given):

Proof: Assume that [\mathbb{N} \rightarrow \{1, 2, 3\}] is not uncountable. This means that there is a surjective function f:\mathbb{N}\rightarrow [\mathbb{N} \rightarrow \{1, 2, 3\}]. We will show that this is impossible by describing a sequence s\in [\mathbb{N} \rightarrow \{1, 2, 3\}] such that s\notin range(f).

Let f_0, f_1, f_2, \ldots be the sequences in the range of f. We define a sequence

s=(g(f_0[0]), g(f_1[1]), g(f_2[2]), \ldots),

where f_h[k] is the k^{th} element of sequence f_h and g:\{1, 2, 3\}\rightarrow\{1, 2, 3\} is a function such that g(i)\neq i for i=1, 2, 3.

By the definition of s, s[n]\neq f_n[n] for all n\in\mathbb{N}, proving that s is not in the range of f. However, s\in [\mathbb{N} \rightarrow \{1, 2, 3\}]. Thus, f is not surjective as claimed, and [\mathbb{N} \rightarrow \{1, 2, 3\}] is uncountable. \Box

A slightly different wording of my solution was included in the solutions for miniquiz two, below a solution which uses the hint given.

I was concerned that 6.042 students would find this significantly more difficult than I, but I dismissed this fear when the rest of the course staff assured me it would be okay. One could say it wasn't too terrible to have put this on the quiz, but let's look at just the results from that problem (Again, compiled by another TA; score values are on the x-axis and the number of occurrences is on the y-axis):

Problem 3 Scores

Some key statistics about the 82 student scores on problem 3:

  • The mean was 2.72 points.
  • The median was 3.00 points.
  • The standard deviation was 2.54 points.

These "key statistics" really didn't set off that many bells; they just indicate that the problem was hard. However, the way the scores were partitioned alarmed us a lot. Additionally, students who have consistently done well on problem sets, on the in class problems, and on the first miniquiz did not correlate with students who did well on this problem. How did this happen?

Students may have been ill-prepared for crafting a solution to this problem. It is true that they have only seen two proofs similar to this problem: the professor's lecture included a proof which uses an argument similar to the one suggested by the hint, and a problem set problem required a proof structured similarly to my solution. It is reasonable to assume that students are not as familiar with proofs presented during lecture as those they have worked out themselves, but this probably does not fully account for why students performed differently on this problem.

Many students who did get at least partial credit on this problem cited that they did not have enough time to think about this problem correctly as they just figured out what they would need to do too close to the end of the quiz. This is probably because the general mindset for 6.042 thus far differs significantly from the mathematical mindset required for proving problem 3's statement, which isn't too surprising given that the mind of a mathematician and the mind of a computer scientist are required to work in (at least partially) different ways. Simply put, problem 3 of the second miniquiz showed me how unnatural theoretical proofs are to computer scientists.

I wonder if this illustrates that a deep understanding of math is becoming less and less important for being a good computer scientist as many successful MIT graduates take no math courses beyond the general Institute requirements and 6.042. However, I'd like to believe that even if this is the case, difficult problems incite curiosity within my students, causing them to dive deeper into mathematics. After all, having a better understanding of mathematics can't hurt them as computer scientists. At least, the questions students have personally emailed to me since getting their quizzes back supports my claim.