What is Wrong with the ACM Typesetting Process?

Recently, I had to go through the process of preparing the camera-ready version for two ACM TODS papers. I am not sure what exactly is the problem but the whole typesetting process at ACM seems to be highly problematic.

My own pet peeves:

Pet peeve A: The copyeditors do not know how to typeset math and they do not even check the paper to see if they have incorporated correctly their own edits.

I detected problems repeatedly and the copyeditor consistently does not check the proofs after making the edits. Here are a few examples.

Example #1

I submit the latex sources and the PDF, with the following equation:


The copyeditor does not like the superscripted e^{\beta x_a}, so decides to convert it into the inline form exp(\beta x_a). Not a bad idea! Look, though, what I get back instead:


To make things worse, such errors were pervasive and appeared in many equations in the paper. I asked the copyeditor to fix these errors and send me back the paper after the mistakes are fixed, so that I can check it again. I get reassured that I will be able to inspect the galley proofs again before they go to print. Well, why would I expect that someone who does such mistakes will be diligent enough to let me inspect again the paper...

A couple of weeks later, and despite all the promises, I get an email indicating that my paper was published and is available online. I check the ACM Digital Library, and I see my paper online, with the following formula:


OK, so we managed to get an interesting hybrid :-). Seriously, do the ACM copyeditors even LOOK at what they are doing? If they do check and they do not understand that this is an error, why do we even have copyeditors?

Example #2

I assumed that the previous snafu was just an exception. Well, never say never. A couple of days back, I got the galleys for another TODS paper, due to be published in the next few days. Again, the copyeditor decided to make (minor) changes in the equations. In my originally submitted paper, I had the following equations:



In the galleys, the same equations look like:


I will repeat myself: do the ACM copyeditors even LOOK at what they are doing? If they do check and they do not understand that this is an error, why do we even have copyeditors?

Pet peeve B: Converting vectorized figures into bitmaps

If you have submitted a paper to a conference, you know how crazy the copyeditors get about getting PDFs with only Type 1 fonts, vectorized, not-bitmapped, and so on. This is a good thing, as the resulting PDFs contain only scalable, vector-based fonts that look nice both on screen and on paper.

For the same reason, I also prepare nice, vectorized figures for my papers, so that they look nice both on screen and on paper. However, for some reason, the copyeditors at ACM they seem to like to convert the vectorized images into horrible, ugly bitmaps that do not scale and look awful. Here is an example of a figure in the original PDF:


Here is how the same figure looks at the PDF that I received as a galley:

Am I too picky? Is it bad that I want my papers to look good?

End of pet peeves

(Note: The same copyediting process, described above, at IEEE seems to work perfectly fine.)

I start believing that the whole idea of publishing is a horribly outdated process. I assumed that copyeditors were a part of a chain that adds value to the paper, not a part that subtracts value.

If I need to check carefully my paper, being afraid that the copyeditor will introduce bugs, that the copyeditor will make everything look horrible, then why do we even have copyeditors? Just get rid of them; they are simply parasites in the whole process! Can you imagine having a professor that teaches a class and at the end the students know less about a topic? Would you keep this professor teaching?

Make everything open access. Let every author be responsible for the way that the paper looks. Let the authors revise papers in digital libraries that have problems. Why we consider perfectly acceptable to have bug fixes and new versions for applications and operating systems, but we want the papers that we produce to be frozen in time and completely static?

Furthermore, the whole motivation for having journals is to have the peer-reviewing process that guarantees that the "published" paper is better than the submitted one. Everything else is secondary. Why keep in the chain processes that only cause problems?

When are we going to realize that the publication system should be completely revamped? Why not having an ongoing reviewing process, improving the paper continuously? Should we keep the system as-is so that we can be "objectively evaluated" by counting static papers that are produced once and never visited again?

OK, venting is over. Back to the SIGMOD papers.

Read more »

Visualizing the Dirichlet

Last week, while working with Foster Provost and Xiahoan Zhang, one of our PhD students, we were trying to understand the internals of the Latent Dirichlet Allocation.

In particular, we were getting strange results from the LDA-C program by David Blei, and we wanted to figure out what we were doing wrong. The first suspects were the parameter values. We wanted to see how the values of the different parameters affect the behavior of the technique.

The conceptual model of LDA is pretty simple. We have a Dirichlet distribution, parameterized by a k-dimensional vector alpha (k is the number of topics). The Dirichlet distribution allows us to draw k-dimensional random vectors theta_i, one for each document in the collection. The vector theta_i represents the topic mixture within the document. For example, for 3 topics (say, art, technology, and sports), a vector theta_i = (0.2, 0.5, 0.3), means that the document i is 20% about art, 50% about technology, and 30% about sports. Then, each topic is modeled as a random distribution of words, representing the relative frequency of each word within the given topic.

The "alpha" parameter of the Dirichlet is crucial, as it defines the distribution of the theta_i vectors or, in other words, the "distribution of topic distributions".

After looking at some visualizations of the Dirichlet, the demonstrated shapes looked strange. Most of the visualizations of the Dirichlet (e.g., on Wikipedia) showed that the mass of the density distribution is around the "center" of the simplex. For example for a Dirichlet with three topics (image from Wikipedia):

most of the density was around the area (0.33, 0.33, 0.33). At the same time, very little mass was allocated at the "corners" of the simplex. Such a shape implied that the majority of the theta vectors drawn from such a distribution would be vectors that have mix many topics within a document. At the same time, very few documents would be generated with only one topic.

This was a counterintuitive modeling choice. Why do we want to force most documents to have many topics and not have the majority of documents to have only a small number of topics? Something seemed wrong.

To understand better how the Dirichlet density function is allocated in the simplex, I pulled my favorite Maple, and started coding my visualization code. First, I defined the 3-d Dirichlet density function.

B := (a1, a2, a3) -> (GAMMA(1.0*a1) * GAMMA(1.0*a2) * GAMMA(1.0*a3)) / GAMMA(1.0*a1+1.0*a2+1.0*a3);

Dir := (x1, x2, a1, a2, a3) -> (x1^(a1-1)) * (x2^(a2-1)) * ( (1-x1-x2)^(a3-1)) /B(a1,a2,a3) ;

Then, I plotted the log of the density in an animated 3d plot. Since I just wanted to see how the value of the vector alpha change the distribution, I decided to keep all the vector elements to alpha equal to each other, and vary them to have values from 0.3 to 2.0.

with(plots);

plotsetup(gif, plotoutput=`LogDirichletDensity-alpha_0.3_to_alpha_2.0.gif`);

animate ( plot3d, [eval(log(Dir(x1, x2, a1, a2, a3)), {a1=a, a2=a, a3=a}), x1=0.00..1, x2=0.00..1, axes=BOXED, grid=[25,25], gridstyle=triangular, orientation=[-135, 60], shading=zhue, contours=20, style=surfacecontour, view=-3..2 ], a=0.3..2.0, frames=100);

The result was pretty revealing:

When the values of alpha are below 1.0, the majority of the probability mass is in the "corners" of the simplex, generating mostly documents that have a small number of topics. When the values of alpha are above 1.0, the majority of the generated documents tend to contain all the topics. (Yes, I am pretty sure that any people with experience in Bayesian statistical modeling who are reading this post are yawning by now. But this is my blog and I will write things that I find interesting.)

So, for general topic collections, and traditional "topical clustering" tasks, it seems best to drive the LDA inference towards configurations that have low alpha values, so that each document contains a small number of topics.

Someone might ask, when we would want at all to examine configurations where alpha is higher than 1.0? Can such configurations be useful at all? Well, there are cases when documents tend to contain frequently a mixture of topics. In product reviews, the reviewers tend to cover a large number of product features (e.g., for a digital camera they will talk about the lenses, about the image quality, about battery life, and so on). Hotel reviews on TripAdvisor also exhibit a similar structure. Similarly, in feedback postings in the reputation systems of eBay and of Amazon, the buyers tend to give feedback about various characteristics of the merchant with whom they transacted (how fast was the delivery, whether the product was accurately described, and so on). In these cases, LDA tends to work best when the values of alpha are above 1.0. Otherwise the resulting topics do not make much sense.

Read more »

Using Facebook ... as a facebook

There is a lot of discussion lately about Facebook, its API, its valuation, its growth prospects, and so on. Lately, I realized that Facebook has a very nice functionality: it is a ... facebook! What do I mean?

I have had a Facebook profile since late 2004. While I have a full profile, I am not that active, since I tend to reach most of my contacts either through email or through instant messaging. Nevertheless, every years students who take my undergrad class, add me as a "friend" on Facebook, and these contacts remain even after the class is over.

Now, as my freshmen students in 2004 reach their senior year, they start asking for recommendation letters. Remembering who is the person who asks for a recommendation, just by looking at the name, tends to be tricky: Each year we deal with hundreds of students, and unfortunately our human memories are not keeping up with Moore's law. However, when the student is on Facebook then I can check easily the corresponding Facebook profile. By looking at the photo of the student, I can remember very easily the student, the performance in class, and the general impression that I had formed during the course. Then, checking the grades in the homeworks and projects completes the image, and writing a recommendation letter tends to be much much easier.

In effect, Facebook for me today tends to be like a LinkedIn for connecting with my students. If only they removed such sentences from their privacy policy: "Facebook may also collect information about you from other sources, such as newspapers, blogs, instant messaging services, and other users of the Facebook service .... We may use information about you that we collect from other sources, including but not limited to newspapers and Internet sources such as blogs, instant messaging services, Facebook Platform developers and other users of Facebook, to supplement your profile..."

Read more »

Only for Database Geeks

From today's xkcd:

Read more »

Implicit and Explicit Changes in Contracts: Rent-A-Coder

I have blogged previously about outsourcing some research tasks using Amazon Mechanical Turk. Another option that I have been using is Rent-A-Coder (RAC): I am using this service mainly for outsourcing basic programming tasks, such as building basic crawlers for data collection. (Even though someone could argue that students can be used for writing such programs, I believe that the time of the PhD students is better spent trying to solve some research problems, rather than completing basic programming tasks). So far, I have been rather satisfied with the overall process, although there were glitches from time to time. (More about the overall experiences in another post.)

However, an incident that happened lately forced me to think seriously about assigning any important project using the RAC platform. Specifically, for one of the projects, we ended up disagreeing with the coder about the specifications. The summary of the disagreement:

• I have posted a project description and the corresponding specifications.
  
• The coder proposed a change to the specifications
  
• I did not accept or reject explicitly the proposed change, but rather pointed the coder to the contract to see the description
  
• I accepted the bid
• The coder had 24 hours to review and accept or reject the project assignment
• The coder assumed that I agreed with the modifications that he proposed, and
  
• The coder delivered the modified project, which was not according to the project specification
  

After the project was delivered, we could not agree whether the deliverable was acceptable or not. Since we could not agree, we resorted to arbitration, which is done by the Rent-A-Coder staff. According the the RAC terms of service, in case of disagreement, the RAC serves as a judge and decides who is correct.

The arbitrator, as part of the analysis said:

The coder has proposed a change in the contract.
The buyer has the following options:
A. Explicitly reject the change in the contract.
B. Implicitly/explicitly accept the change in the contract.
The buyer did not reject the change and implicitly accept the new requirements.

These steps are not described in the terms of RAC, but rather are devised by the arbitrator. In defense of this ruling, the arbitrator said that the same rules would apply if myself, as a buyer, had asked for some extra features to be implemented. In such a case, if the coder did not explicitly reject them, the coder would have to implement the extra features, even if they are not described explicitly in the project description.

And here lies my disagreement with the ruling. My understanding is that a contract can be modified only explicitly, not implicitly. A party can reject changes implicitly or explicitly. This setting effectively gives priority to the statements in the contract (in this case, project specification) over the changes that are proposed during negotiations (which can be accepted and agreed upon explicitly). Otherwise, we have a dangerous precedent, where one party (buyer or coder) can start proposing an endless list of amendments, and the other party has to waste time explicitly rejecting the proposed changes, stating that they are out of the scope of the original project description.

At least this is my understanding of the Uniform Commercial Code. It is clear to me that the Rent-A-Coder has the right to rule independently of the provisions of the code. However, it seems problematic to devise a new set of undocumented rules when handling outsourcing contracts, instead of relying on existing legislation. If not anything else, it does not build trust among the participants in the RAC marketplace to know that the existing law does not apply in the RAC contracts.

Opinions? Am I incorrect in my analysis?

Disclaimer: I am not a lawyer, so the analysis above is based on my own interpretation of the law.

Read more »