1Overview

1.1What is research?

"To research" is to search closely, and "to search" is to seek or wander.123

To research is to accumulate knowledge.

To research is to answer a research question or solve a research problem.

Meta-research is research about research.4

Scientists and academics are researchers, but not always the other way around.

In practice, a theoretical research is mainly trying to answer focused (deep and narrow) questions.

1.2How do we research?

This is how we do research:

  1. Think about an idea we are interested in.
  2. Find related publications.
  3. Summarize how those publications are related to our idea.
  4. Write our article.

Ask "What questions should we ask?"

Advice?5

What is an ideal research process, in BPMN (Business Process Model Notation)?

Researchers need something like a global version of a combination of Wikipedia and AirBnb's Knowledge Repo6.

Ask research questions, answer them, and collect them into living documents.

1.3What is the best place to do research?

Find the leading researcher, and go there?

There are some disparate systems. I wish there were one system for all scientific publications.

To find papers and citations, use Semantic Scholar7 or Google Scholar8. Semantic Scholar is much richer than Google Scholar. Semantic Scholar enables us to find out who influences who, what cites what, and which papers are similar, and much more. It is the closest thing to my dream of having an artificial-intelligence that assists my research. It makes Google Scholar look like a dumb pile of data. A weakness of Semantic Scholar is that, as of 2019, its notion of "the same author" is somewhat lacking: sometimes it thinks "John Smith" and "J. Smith" are two different authors.

What is Iris AI9?

Libraries?10

What is IGDORE?11

Sometimes telephones may be necessary.12

Use Google to find papers. We may also add filetype:pdf to your query.

These are disparate systems. To find theses and dissertations, use OATD13 (open-access theses and dissertations). To find a UK PhD thesis, use EThOS14 (e-theses online service). Deutsche Digitale Bibliothek - Kultur und Wissen online (English available)15

How do we know what others have done? I think, today, in a research group, there has to be a "lead researcher" dedicated to finding resources (papers, books) that might be related to the group's research, because this job is huge, given today's flood of information.

Suppose we have found a related article. Which should we zoom into: the author who writes it, or the venue where it is published?

Both an author and a venue have stable narrow interests. But only authors can ask questions or solve problems. Therefore, the author is more important than the venue, and we should check the author's publication list before we check the venue's publication list. Therefore, when we come to a conference, we should aim to make friends with people with similar interests. Do not attend a conference that has a topic you may be interested in, but do attend a conference that may be attended by people who share your interests. The source of an idea is the author, not the venue. The thought lives in an author, not in a venue.

Why do people have stable interests? Adults have stable interests because adults rarely change beliefs.

People do what they think are important for them. It is wonderful to find another person who also thinks that what we think is important.

There are too many journals for authors to compare. Authors publish to where their friends publish. Thus it is important that people with similar interests know each other.

<2018-10-02> Twitter is surprisingly good for computer science research?16

Finding recent publications using academic search engines?

I tested Google Scholar17, Microsoft Academic18, and Bielefeld Academic Search Engine (BASE)19, using these queries: "partial evaluation", "optimal lambda".

BASE can boost open-access documents.

Are there academic metasearch engines / search aggregators?

Why do different fields have different search engines? What is the difference between biology literature study, computer science literature study, and physics literature study?

2.2The problem of multiple authors

Usually the last author is a supervisor, a professor, a research director, or something like that. Thus the true interest of the last author is masked.

A publication only truly reflects the interest of the first author.

The first author contributes the most to the publication.

2.3How do we traverse citations?

2.3.1What is a citation?

In the 15th century, "to cite" means "to summon".20 In 2018, "to cite" means, roughly, to hyperlink.

In 2018, a citation is a printable hyperlink21. Citations and hyperlinks serve the same purpose: to link information. We cite for the same reason we hyperlink.

A citation is a reference to external content.

A citation is a reference that is too distracting to inline.

CiteSeer22 is an automatic citation indexing system[2].

2.3.2Why do we cite?

A citation indicates that the citing author thinks that the citee is somehow related to the citer.

Why do we cite? What is the relationship between a citer and a citee? One answer is that the citer cites the citee because the citee is interesting[3]. Another answer is implied by the Citation Typing Ontology (CiTO)[4], which was designed for biomedical research, but should also be applicable to other fields. Another answer is implied by a taxonomy of the motives of citing[1].

Citations may be fraudulent. Researchers are humans.

2.3.3Other points of view

There are three points of view: the citer, the citee, and the reader.

Most ideas are a sentence in first-order logic. Two sentences \(p\) and \(q\) are compatible iff they don't contradict each other. They are compatible iff their conjunction is not false.

How come there can be a case where we don't know the truth of \(p\) and we don't know the truth of \(q\) but we know that \(p \wedge q\) is false?

The strength of a claim is determined by the difference between the strongest supporting argument and the strongest opposing argument.

2.4How do we know the current sum of human knowledge?

This is hard. I don't know.

Finding other information sources: Finding other wikis23.

Science 2.024, Open Research25, Open Science26. Minimize the duplication of work.

2.5Which conference should we attend?

The goal of attending a conference is to find people who share similar goals with us. Thus, we should not see what the conference is about, but who attends the conference.

2.6Where are expository works?

MO 15366: Which journals publish expository work?

These journals2728 contain expository works. Unfortunately they are Elsevier journals.

Arxiv math history and overview29

2.7How do we use journals?

  • What are the journal article types? What kinds of articles do journals publish?

    • Editage's six journal article types: original research; review article; clinical case study; clinical trial; perspective, opinion, and commentary; book review
    • There are also letters, monographs, surveys, and retractions.

3Summarizing publications

3.1How do we read a research article?

Read the title, the abstract, and then the conclusion.

3.2How do we read a book?

Do not read the entire book.

Get an idea or a problem before touching the book. Find relevant parts in the table of contents.

3.3How do we catch up quickly?

An article is a diff. It is a patch. To understand the article, you need background knowledge. We are weak in bootstrapping new people quickly.

If you are in a lab, you can ask your professor to bootstrap you.

If you are an independent researcher?

Only read papers or books after you know what you are looking for. Don't read without reason.

3.4How do we onboard newcomers faster?

Write living documents that summarize papers and provide directions. Let newcomers improve them. Living documents are perfect for expository works.

Every research paper is a "delta" of knowledge, a "nugget", a "diff". We need living documents that are the sum of those deltas.

A possible problem: who are the authors of a collaborative living document?

3.5How fast can we speed-read?

4Writing

4.1What computer document format should I write my research in?

Write your content in Pandoc Markdown / Org Mode. Let Pandoc convert whatever format to HTML or LaTeX.

Use MathJax if you need to typeset math on the Web.

Stick to standard Web technology. For the reader, reading HTML pages has less friction than reading a PDF document.

Use CSS to style the HTML.

Why should we not follow my advice? Your institution probably incentivizes you to publish on famous journals, not on the Internet. This incentive scheme has to change for better science. Unfortunately you are not in the position to change it. It's frustrating.

4.2How do researchers collaborate?

Timothy Gowers: Is massively collaborative mathematics possible?

4.3How should we write?

Slides, Simon Peyton-Jones, "How to write a great research paper: seven simple suggestions"30. Another copy of those slides is at Microsoft31.

4.4What writing medium should we use?

Researchers should collaborate writing living documents instead of writing isolated papers. We need something like Wikipedia but for research. We need a goal-oriented/task-oriented Wikipedia. Wikiversity might be close to that.

5Entering academia

The most important thing when entering academia is to find a professor whose research interests match ours.

If we want to enroll in higher formal education, then we must find a professor whose research interests match ours, and we must prove that we are worthy of the research position. The right professor is the most important factor.32 The professor is more important than the institution (university or company). We must be prepared to relocate to somewhere near the professor.

https://www.quora.com/How-do-professors-view-cold-emails-asking-for-research-opportunities

6Academic journal publishing reform

6.1What

Springer makes no fucking sense: personally spending $39,00033 per year just to read papers is just too fucking much for an average researcher.

Publishers may be evil.34

6.2Proposal for academic publishing reform: Separate peer review and physical dissemination

Attach reputation to reviewers, not journals. To humans, not institutions. What is important is not where an article is published, but who reviews the article. There are no important journals. There are only important reviewers.

Alternatively, make the reviewers own the journals. Profit-seeking is not evil on its own. Neither is monopoly. But together they screw buyers.

Incentivize academics to publish openly on the Internet instead on closed journals.

Make a website for peer-reviewing articles.

2018-04-25: It exists. See Wikipedia: Publons, PubPeer, JournalReview.org.

6.3Other content

6.4Academic grant funding is broken

Researchers waste time adorning grant proposals in the proposal arms race. We should use some randomness.35

7.1Reading academic publications

  • The input is the publication and some effort.
  • What is the output?

  • What is the best way of reading?

    • An attempt to answer that

      • Begin with a goal.

      • Every time you read a sentence, compare it to your goal.

        • If the sentence helps you reach your goal, process the sentence.
        • Otherwise, skip it.
  • A publication can be thought of as a set of first-order logic statements.

  • What is knowledge?

    • A piece of knowledge is a sentence: a first-order logic statement with no free variables.

7.2How should I structure my research?

  • Structure your research as a hierarchy of questions and answers. Every question may spawn child questions and answers. I call this format the HQA (Hierarchical Questions and Answers) or QAT (Question-Answer Tree).

    • We should group paragraphs into trees.

      • Every child explains, augments, supports, or elaborates its parent.
      • We have been grouping sentences into paragraphs.

  • Why do we need questions or problems?

    • Questions drive research.

7.3How should we organize information?

7.4Journals

7.5Exploration

7.6How do I keep myself up-to-date? How do I keep myself in the loop?

7.7How do we know if something is legitimate or bullshit?

7.8What tools might help my research?

7.9Open access journals

How do we know if an open-access journal is legitimate?

http://www.mdpi.com/journal/universe

https://benthamopen.com/PHY/home/

7.10How do we work with proofs?

7.11Can we crowdsource/outsource master-level or doctorate-level research to Fiverr or Amazon Mechanical Turk?

I think no. Thinking doesn't scale. Some data collection may scale.

Trying to scale the thinking may invite crackpots?

7.12Can we transform or break a doctorate-level research problem into pieces that amateurs can work on?

Is the difficulty in mathematics essential or accidental? For example, accidental complexity is due to bad notation, bad presentation, bad writing.

7.13What are the different kinds of academic meetings?

Which is the most common type? Which one should I attend?

"Conference" comes from Latin "con-" ("together") and "ferō" ("I bear")36

"Colloquium" comes Latin "co-" ("together") and "loquor" ("to talk")37

seminar

congress; Wikipedia: International Congress of Mathematicians

See also English SE 20924.

"Symposium" comes from a Greek word meaning "to drink together".38

What is the difference between conference, congregation, symposium, seminar? Why can't we just use the word "meeting" for all of them?

7.14Why is literature study important?

  • It saves time.

    • Skip things that are already done.
    • Skip dead ends.
    • Build on other people's work.

7.15How should we structure human knowledge?

7.16Undigested

7.17Scientific method?

7.18Failures?

Publishers that fail the SCIGen test?

Lupine Publishers39

Other failures?

"Peer review fails to prevent publication of paper with unsupported claims about peer review"40

I read somewhere I forgot, that that is not the point of peer review. What is peer review meant to accomplish then?

7.19How to study everything?

Begin with ontology. What exists? How do they relate?

7.20Open-access journals?

8.1How do we get research grants?

What are the laws and rules?

Who makes the decisions?

How do you improve your chances?

2014 book "Writing Successful Grant Proposals from the Top Down and Bottom Up"41

8.2What does an assistant professor do?

According to https://recruit.ucsc.edu/apply/JPF00651:

  • develop a research program
  • advise graduate students in their research area
  • obtain external funding
  • develop and teach courses within the undergraduate and graduate curriculum
  • perform university, public, and professional service

That's a lot for one person to do.

8.3Problems

  • 2005, article, "Why Most Published Research Findings Are False", John P. A. Ioannidis, html

8.4Academics is broken. Science is broken. Research is broken.

  • https://www.statnews.com/2018/10/29/alzheimers-research-outsider-bucked-prevailing-theory/
    • Alzheimer research went slowly. 30 years and still no cure. Because grants are decided by narrow-minded old high guards. Perverse incentives. Too narrow. Too incremental. Too risk averse. Too tribal. Groupthink. Science has become a religious monoculture.
      • Science requires diverse thoughts in order to progress.
  • Theoretical physics is also broken. It's hard to get a grant if you don't do string theory. See Lee Smolin's commentary.

8.5Academic writing

  • "How to write a good CVPR submission", pdf slides

8.6Math PhD?

  • 2011, article, "A Very Rough Guide for PhD Students in Mathematics", pdf

8.7Mathematical knowledge management?

8.8Library problems

In 2018, a lot of human knowledge is still in libraries and not available on the Internet. Digital typesetting was invented relatively recently. Papers should be scanned and OCR-ed. AI can help.

Why are there different libraries? There should be only one library in the world. All library should be the same. All library should have the same content.

8.9Wikipedia alternatives?

8.10Refuting bullshit

  • WP:Bullshit asymmetry principle
    • "The amount of energy needed to refute bullshit is an order of magnitude bigger than to produce it."
    • Does that mean it's easier to fight a bullshit with another bullshit?
      • Why don't we fight bullshit with bullshit?
      • Why don't we fight false news with false news?
      • Why don't we fight hoaxes with hoaxes?
      • Why don't we fight propagandas with propagandas?
      • If we fight lies with lies, everybody loses?

9Living their dreams doing research?

The Max Planck Society http://elevanth.org/blog/2018/09/02/golden_eggs/

10Bibliography

[1] Erikson, M.G. and Erlandson, P. 2014. A taxonomy of motives to cite. Social Studies of Science. 44, 4 (2014), 625–637. url: <http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.859.95&rep=rep1&type=pdf>.

[2] Giles, C.L. et al. 1998. CiteSeer: An automatic citation indexing system. ACM dl (1998). url: <https://clgiles.ist.psu.edu/papers/DL-1998-citeseer.pdf>.

[3] Liu, Y. and Rousseau, R. 2013. Interestingness and the essence of citation. Journal of Documentation. 69, 4 (2013), 580–589. url: <https://www.researchgate.net/publication/263682305_Interestingness_and_the_essence_of_citation>.

[4] Shotton, D. 2010. CiTO, the citation typing ontology. Journal of biomedical semantics (2010), S6. url: <https://jbiomedsem.biomedcentral.com/articles/10.1186/2041-1480-1-S1-S6>.

  1. https://www.etymonline.com/word/research↩

  2. https://www.etymonline.com/word/search↩

  3. https://en.wikipedia.org/wiki/Research↩

  4. https://en.wikipedia.org/wiki/Meta-research↩

  5. https://cstheory.stackexchange.com/questions/2953/advice-on-good-research-practices↩

  6. https://medium.com/airbnb-engineering/scaling-knowledge-at-airbnb-875d73eff091↩

  7. https://www.semanticscholar.org↩

  8. https://scholar.google.com↩

  9. https://iris.ai↩

  10. https://medium.com/a-wikipedia-librarian/youre-a-researcher-without-a-library-what-do-you-do-6811a30373cd↩

  11. https://medium.com/@IGDORE/new-academia-a-safe-harbour-for-researchers-who-love-science-c4baa87c1ebe↩

  12. https://academia.stackexchange.com/questions/73236/how-to-find-people-who-do-similar-research↩

  13. https://oatd.org/↩

  14. http://ethos.bl.uk/Home.do↩

  15. https://www.deutsche-digitale-bibliothek.de/?lang=en↩

  16. https://twitter.com/search?q=programming%20language%20research↩

  17. https://scholar.google.com/↩

  18. https://academic.microsoft.com/↩

  19. https://www.base-search.net/↩

  20. https://www.etymonline.com/word/cite↩

  21. https://en.wikipedia.org/wiki/Hyperlink↩

  22. http://citeseerx.ist.psu.edu/index↩

  23. https://en.wikipedia.org/wiki/List_of_wikis↩

  24. https://en.wikipedia.org/wiki/Science_2.0↩

  25. https://en.wikipedia.org/wiki/Open_research↩

  26. https://en.wikipedia.org/wiki/Open_science↩

  27. Expositiones mathematicae https://www.journals.elsevier.com/expositiones-mathematicae/↩

  28. Computer science review https://www.journals.elsevier.com/computer-science-review↩

  29. https://arxiv.org/archive/math.HO↩

  30. https://www.cis.upenn.edu/~sweirich/icfp-plmw15/slides/peyton-jones.pdf↩

  31. https://www.microsoft.com/en-us/research/academic-program/write-great-research-paper/↩

  32. https://www.quora.com/What-are-the-good-complexity-theory-research-groups-around-the-world↩

  33. 5 paper/work-day * $30/paper * 260 work-day/year = $39,000↩

  34. https://www.theguardian.com/higher-education-network/2015/sep/04/academics-are-being-hoodwinked-into-writing-books-nobody-can-buy↩

  35. "Researchers spend much of their time writing grant applications, which adds up to a lot of wasted resources. Is there a better way to fund science?" https://www.youtube.com/watch?v=JkMvm_rsTvI↩

  36. https://en.wiktionary.org/wiki/conference↩

  37. https://en.wiktionary.org/wiki/colloquium↩

  38. https://en.wikipedia.org/wiki/Symposium↩

  39. https://www.sciencealert.com/how-kim-kardashian-inventor-bitcoin-teamed-up-write-scientific-paper-satoshi-nakamoto-predatory-journals↩

  40. https://scholarlykitchen.sspnet.org/2018/03/15/a-comment-on-klein-et-als-comparing-articles-to-preprints/↩

  41. https://us.sagepub.com/en-us/nam/writing-successful-grant-proposals-from-the-top-down-and-bottom-up/book236652↩