Introduction

I have been thinking of describing the way I read a paper. Although modern AI tools, especially LLMs, are designed to do such tasks very successfully, I want to express that there is still some value only human professionals can deliver (hopefully). In academic research, publications such as conference papers are just the outcome under the spotlight. It is not easy for others to understand how much effort it took and how to read a paper effectively and efficiently in a way that the manuscript can actually inspire the reader as designed. From a superficial level, papers are just articles, or literature. The general public can have a certain level of understanding because usually it is written in English, one of the most commonly used natural languages in academia. However, how to read deeper and be able to extract more detailed information, implications, and inspiration the paper tries to deliver is not trivial at all. One simple reason is that it requires background knowledge. It’s more like the context you should already know, or at least know how to know, in the relevant domain(s). Therefore, reading an academic paper is not something like reading a news article. You also need to consider the trade-off between reading speed and depth of your reading comprehension. Reading a paper for the first time might take you a week. I had a similar experience where I didn’t know how I could understand a paper.

I always think that reading academic papers intimidates people who want to step into this community, while professionals feel it is such an easy and routine task. Hopefully this writing can point out the gap. Not to mention that the language barrier is another layer of reading challenges for non-native speakers like me. This is a very subtle thing. Sometimes you can’t be sure whether the reading difficulty comes from the language barrier or the reading material itself. That’s why scientific paper writing pursues objectivity, logical reasoning, avoiding adjectives and things of that kind. You would probably never know whether the same manuscript written in either your native language or the authors’ (if that is even worth doing, or possible to retain every subtle piece of information and implication and such) would make the reading comprehension different.

Table of Contents generated with DocToc

• Introduction
• Disclaimer
• Where to Start
• Step 1: Read the Title
• Step 2: (Optional) Read the Author List
• Step 3: Read the Abstract
• Remarks

Disclaimer

I do not intend to teach any “correct” way to read a paper, nor suggest that all my analyses here are correct or complete (or better than other similar works), or free of misunderstanding. And I cannot guarantee this example paper has no oversights, mistakes, plagiarism, and so on. This writing is only my personal opinion and understanding, and hopefully is somewhat beneficial for some people. We will see an AI summary of the same paper at the end of this writing^*. Another thing is: this writing is casual. I do not intend to make my writing so-called “high-quality” as an academic paper. Guess what? I still have to use the LLM tool to revise/grammar-check my writing here, as a satire of the LLM era. If you think AI can deliver what I intend to do here, feel free to do so. This is just an organic human writing material that tries to “decode” the paper.

Where to Start

I intentionally skip the discussion on how to choose a paper to read, which is another topic. I simply picked one of the most interesting and impressive papers I have seen so far in the domain of computer systems. PanguLU^[1] impresses me with the eyeball-catching figure on the first page that visually explains the whole idea of this paper. One important question we should ask when it comes to reading a paper: Why should I read this particular paper? I think reading a paper is mostly about the utility, not like reading news that is just for fun or other recreational purposes. Of course, reading a paper could be as fun as reading other forms of literature, if you enjoy it. And my academic background is computer architecture. I will try to demonstrate my knowledge and skill on how to comprehend this paper (again, in a casual way). So, let’s assume that you were interested in reading this paper, and you want to know: what can I learn from this work? And you are somewhat prepared to devote some time to read this paper. During the reading process, you may need to study some other background knowledge, or take some time to digest some context. It might be easier for you to open up the paper itself sitting by the side of this writing.

Step 1: Read the Title

The title is usually designed deliberately to express the most important idea, concept, structure, solution, or any other relevant content in just one sentence. And it should give you a sense of whether this work is relevant to you, interesting to you, or requires any background knowledge to read. “PanguLU: A Scalable Regular Two-Dimensional Block-CyclicSparse Direct Solver on Distributed Heterogeneous Systems” says a lot of important information. First, scalable. Scalable is one of the contributions of this work. Why? Because sparse linear solvers usually suffer from not being scalable. Unlike a dense linear solver that is straightforward, a sparse linear solver may achieve better computational performance in exchange for scalability. Sparsity pattern recognition and exploitation is the main challenge. The fourth contribution bullet point at the end of the introduction section: “We achieve significant speedups … and prove that PanguLU scales well on ….” does mention “scales well”. The next part: “Regular Two-Dimensional Block-CyclicSparse Direct Solver” describes the high-level structure of the solution (a direct sparse linear solver). And due to such blocking design, tasks associated with the blocks can be assigned across heterogeneous computing cores on a “Distributed System”. By now you could gauge how interesting this work is to you, and whether you should read the rest of this paper. In practice, you may want to additionally read the abstract as well before actually deciding whether to keep reading this paper.

Step 2: (Optional) Read the Author List

This step may sound tricky. Why would knowing who the authors of this work are matter? Isn’t knowledge universal and objective? After all, we shouldn’t judge the merit of a scientific work based on who contributes. (If that makes sense.) Nonetheless, this step helps you construct social understanding. You may recognize the authors whose research topics/areas you know they are working on, so that you have a sense of the taste of this work. Or that may even help you gauge how important and novel this work could be. This is a practical step, not scientific at all, that I think no researcher would deny doing. If you are not the reviewer who reviewed this paper to determine the publication, you are supposed to know this paper for the first time. Excluding coincidence or any other reasons, you will probably build an impression about the people and their affiliations in your mind. That affects the way you look at other papers related to them or their institutions. This is a social behavior. Think of it: how a paper gets accepted is based on human judgement (hopefully people, humans, still believe in it and practice it). The irony is that people think scientific papers are objective, while the publication process relies on human decisions. There is no true standard. My belief is that public scrutiny is the regulator, and it shouldn’t only belong to domain experts.

Step 3: Read the Abstract

The abstract mostly works as a short summary of the whole paper. It usually describes what the problem this work tries to address is, challenges of working on the problem, and an outline of the solution. Experimental results are usually there at the end. This is the convention people follow, sort of like an AI summary nowadays but prepared by the authors themselves. The abstract is the trailer of a paper.

“Sparse direct solvers play a vital role in large-scale high performance computing in science and engineering.” This is a good starting sentence. It is straight to the main point of what this work is talking about (topic), and raises attention by mentioning the importance of caring about sparse direct solvers. This is a one-sentence motivation. After all, readers care about utility. And arguing why this work is important and worth reading is the job of the authors. The earlier they touch on this objective, the better. Otherwise, people may lose attention just like when watching TV.

“Existing distributed sparse direct methods employ multifrontal/supernodal patterns to aggregate columns of nearly identical forms and to exploit dense basic linear algebra subprograms (BLAS) for computation.” This section describes some existing methods that leverage multifrontal and/or supernodal patterns. It gives you a quick reminder about this important background knowledge that you should have in your mind, since later on they will be reasonable baselines. The clever design is that this supernodal knowledge can be reviewed in Figure 1(d). Placing this huge Figure 1 before the abstract is rare while creative. Now you want to either (1) keep reading the next sentence of the abstract, or (2) go to review Figure 1 or even browse the related background knowledge online. Since it’s still early in reading the abstract, I will hold that thought and choose (1) for constructing a complete and basic understanding.

“However, such a data layout may bring more unevenness when the structure of the input matrix is not ideal, and using dense BLAS may waste many floating-point operations on zero fill-ins.” It is almost equivalently important to mention challenges besides what the topic is. Because “challenges” expresses why existing works and solutions cannot solve the task well. And that implies why this work will be novel, worth reading, and valuable. Oftentimes, the challenges will be addressed in a new way that existing works often overlook. Elaborating a new angle on challenges, or re-stating a commonly recognized “challenging” challenge, works. “not ideal” implies that the sparsity pattern of a linear system matrix is challenging to recognize and capture. Existing works may have some ways to capture certain patterns well, but not any arbitrary kinds of patterns. Multifrontal and/or supernodal methods have a reordering phase as Figure 1(b) illustrates that changes data layout. “unevenness” implies that there exists a method to measure evenness quantitatively. You may start to think about any method you know; you can imagine whether it is fair, or expect the author to explain the measurement somewhere later. But the interesting thing is, unevenness could be argued qualitatively because visual judgement is convincing. You can compare subfigures of Figure 1. So how to evaluate the claimed challenge could be subjective.

This paper makes a good expansion on explaining this challenge with a clear motivation section. Later on when you read subsection 3.1 ”Uneven Block Sizes” and subsection 3.2 “Redundant Zero Fill-ins” you will have a better understanding. Conventional methods that use fixed dense BLAS kernels for computing such blocks with uneven sizes will face the performance degradation issue (such as “the computational and storage efficiency”) due to the irregularity. These methods also create additional but redundant zero fill-ins. And we know a simple fact that zero * x = zero. Same as zero + x = x. This will make computations in vain. And this is the key reason why sparse computing research work is useful. After all, these research works are more about computational efficiency, not correctness. Thus, this sentence in the abstract implies improving computational efficiency is critical, which you should already know when you read the title.

“In this paper, we propose a new…” This whole paragraph can be viewed as a whole for now. Either you have the ability to quickly absorb the proposed solutions and make connections between challenges and solutions, or you can simply recognize this fact (assume it is true) without actually thinking it through. Same for the last paragraph: “Experiments on…” that gives you a few critical and high-level quantitative results from their experiments. People use these numbers to argue the value of this work. How much can it improve over the baseline? So far you don’t have further evidence to evaluate how good this work is. The numbers here alone usually can’t express the merit. The discussion has to be expanded as at the bottom of the paper, usually in the experimental results section. But at least by now you get the sense that PanguLU is a good work. And so far the abstract follows a very common template.

How fast should you finish reading the abstract? These 225 words require about 1 minute. But your actual reading speed also depends on how much technical background knowledge you have. The whole writting here as a monologue may take you about 15 minutes to read, assuming that all this makes sense to you. It works as a bridge to show you why there is a reading speed difference between people. Because reading comprehension is about speed and also understanding the context well. You can always claim that you are the fastest calculator person in the world for any mathematical problems, if you don’t care about correctness.

Remarks

Reading an academic paper takes time. It’s like any other ability or skill that professionals always claim is not hard; it is the goal you should achieve, but currently you may just not be able to do so. That frustrates you. The paper is there; accessibility is not a problem. But how much you can absorb after reading is. At an extreme surface level, you may only be able to enjoy the diagrams as painting art, seeing text as purely random ink spraying. What a funny thing. Considering that reading long-form content has become an issue nowadays, and also I’m talking about how I read a paper, I think that keeping this writing not too long is also adequate. Imagine that I’m going to decode the PanguLU paper; who knows how long the complete commentary will be. So I plan to split it into multiple parts. Another thing is, I believe “Pangu” here refers to the primordial figure in Chinese mythology and Taoism. It creates the world by separating heaven and earth. The aesthetic Figure 1 in this paper shows the feeling of making the world organized, creating vitality within. Very impressive.

[1] Fu, Xu, et al. “Pangulu: A scalable regular two-dimensional block-cyclic sparse direct solver on distributed heterogeneous systems.” Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. 2023. https://dl.acm.org/doi/10.1145/3581784.3607050

[*] No, we don’t. Not now.