1Science and engineering

1.1Engineering as applied science

Engineering is the application of science.

Engineering consists of design, modeling, calculation, application of science, and turning all those plans into reality.

Engineering is tradeoff, compromise, optimization with respect to constraints, such as physical constraints and economic constraints.

Engineering implies design goals, tradeoffs, and decisions.

Engineering does not inherently involve many people. A one-man engineering project is an engineering project. It is the methodology that makes it engineering, not the amount of people involved in it. However, when many people are involved, some methodology has to exist in order to ensure that any progress is even possible at all.

To engineer something is to scientifically force something.

Inability to maintain our past works is a sign of engineering failure. A person is actually three people: his past self, his present self, and his future self.

"Science" simply meant "knowledge".1 What is knowledge? There are at least two theories of knowledge:

  • Popper: science is a falsifiable belief. That means, the belief must be such that it is possible to design an experiment that disproves the belief.2
  • Plato: knowledge is justified true belief

(Do we need to dive into epistemology here?)

What is a scientific theory? A scientific theory is a theory that is falsifiable, has explanatory power, and has predictive power.

1.2What does an engineer do?

A civil engineer plans the materials according to the physical load that the structure is expected to withstand. The software engineer's analog is capacity planning of the computational resources according to the computational load that the software is expected to withstand.

2Software engineering

2.1What is software engineering?

X engineering involves the design of X.

Material engineering involves the design of materials.

Chemical engineering involves the design of chemicals.

Civil engineering involves the design of civil structures.

Software engineering involves the design of software.

2.2What is software engineering?

2.2.1Ontology and jargon

A language is something that has syntax (form) and semantics (meaning).

A language enables its users to communicate with each other.

A language is formed by the agreements of its users.

A formal language

A programming language is a formal language that is used for programming (a machine).

To program a machine is …?

A program is a formula/string/term in a programming language?

A machine can be modeled by a formal system.

A program is an executable mathematical object?

2.2.2<2018-09-25> Software is broken and bloated

2.2.3Why is hardware more standardized than software?

Hardware standardization benefits everyone (both its makers and its users). Hardware unstandardization is more expensive than hardware standardization.

Software standardization is more expensive than software unstandardization.

Is software too malleable?

Hardware engineers have standard nuts, bolts, connectors, and components.

Why standardize?

  • standardization reduces cost
  • standardization reduces duplicate effort
  • standardization improves interoperability

Hypothesis: Programming language is too free. If programming language is limited (that is, if there is only one way to do one thing), then software will standardize itself.

There are some software standards, such as ABIs (Application Binary Interfaces). C calling convention is a standard.

2.2.4How do we write lasting software?

  • Clear ontology
  • Task-oriented documentation
    • Documentation doesn't have to be separate from code.
  • Capture the requirement. Translate the requirement into software, instead of making a software system that satisfies the requirement.
    • The program may be rewritten, but the functional requirements, the business logic and the mathematical truths rarely change.
      • In order to make software last, we must capture the requirements, not make a system that satisfies the requirements. This requires a change in mindset. The software is not something made to some requirements; the software is the requirements itself. The software is not the shirt made-to-order; the software is the order itself, from which all satisfying shirts can be generated/derived/produced.
  • Think for yourself. Don't just follow hypes. Always have a reason when doing things. Record your reasoning for your forgetful future self.
  • Design software to be refactored without surprises.
    • Example: Avoid runtime reflection (unless for the most obvious cases). An alternative to that is code generation.
      • Help the compiler help you.
      • Even the most obvious cases aren't obvious to someone who see the code for the first time?
  • Shorten the edit-compile-test cycle. Design software so that it compiles fast and it starts up fast.
  • What else?

2.2.5Complexity

  1. Renaat Verbruggen: Three universal methods of reducing complexity: partitioning, hierarchy, independence

  2. How complex systems fail?

    • 2000, article, "How complex systems fail", pdf

2.2.6Defining software complexity as the length of the shortest first-order-logic formula that describes it?

Every software can, in principle, be described by a first-order-logic formula that describes the requirements satisfied or behaviors exhibited by the system.

  1. What is complexity?

2.2.7Unpopular opinions

2.2.8Who reads SWEBOK? How can we read SWEBOK? Who reads all of it? Why does anyone bother writing it?

Fowler 20034 points out the problems with SWEBOK.

http://beamphys.triumf.ca/info/SWEBOKv3.pdf

2.2.9Coupling

  • Coupling is deciding what you can depend on.
    • Big Company X just released this new cool stuff, but will it still be there in 5 years?

2.2.10Testing

  1. If proving is cheap, prove it instead of testing it.

  2. Amount of testing should be proportional to risk.

    • How do we make code testable?
      • Minimize its dependencies.
      • Minimize the number of things that you must set up before you can run it.
      • Why?
        • If you don't satisfy all its dependencies, you can't run it.
        • If you can't run it, you can't test it.
        • If your business logic depends on the database, you'll have to set up a database to test your business logic.
    • What do we need to test?
      • Testing is proportional to risk.
        • Riskier code should be tested more.
          • "Risk comes from not knowing what you're doing." (Warren Buffett)
            • Buffett said that for investing, but it also applies to programming.
      • If it's obvious, don't test it. (Getters, setters, and DAOs without fancy logic)
      • If you can prove it, don't test it.

2.2.11TODO Compare programming language research journals and conferences

Which ones should we care about? Which ones are the real deal?

  1. Conferences

    • ICFP: The ACM SIGPLAN International Conference on Functional Programming
    • OOPSLA: ACM SIGPLAN Object-Oriented Programming, Systems, Languages & Applications

    https://en.wikipedia.org/wiki/OOPSLA

    • PLDI: ACM SIGPLAN Programming Language Design and Implementation

    http://www.sigplan.org/Conferences/PLDI/

    • POPL: ACM SIGPLAN Principles of Programming Languages

    http://www.sigplan.org/Conferences/POPL/

  2. Proceedings

    • PACMPL: Proceedings of the ACM on Programming Languages
  3. DBLP vs Semantic Scholar?

    https://dblp.uni-trier.de/

2.2.12Software engineer associations

2.2.13Software business model

  • Hardware is moved.
  • Software is copied, not moved.
  • When you sell hardware, you transfer ownership. The hardware doesn't multiple into two.
  • There are two models:
    • SaaS (software as a service, a web application; you control the hardware that runs the software, but not the input and output of the process)
    • bundling software with restricted hardware (Apple iPhone)
  • Controlling software requires controlling the hardware that runs it.
    • Controlling the law requires controlling the lawmakers and the law enforcers.
  • Isn't it absurd to judge a crime by its potentiality not its actuality?
  • Process is software. Law is software.
  • The chemical process in a chemical plant is also software, but not everyone owns a chemical plant.
    • If you hire a chemical engineer to design and implement your chemical plant's process, does he own the rights to the process?
    • If you hire a software engineer to design and implement your computer's operating system, does he own the rights to the operating system?

Ownership is the ability to control access. To own a thing is to be able to control the access to that thing.

2.2.14Measuring software quality?

2.2.15Technical writing, software documentation

"Documentation needs to include and be structured around its four different functions: tutorials, how-to guides, explanation and technical reference. Each of them requires a distinct mode of writing."

2.2.16TODO Who said this "pit of success"? It was someone on the Internet.

  • Write code that makes your client/user/reader (another programmer who uses your code) fall into the pit of success.
  • Corollary: Principle of least astonishment.
  • Corollary: Write code that a programmer can understand with an IDE without having to read much documentation.
  • Corollary: Group data and methods that need those data together in one class.
  • Can that principle derive these principles? http://c2.com/cgi/wiki?PrinciplesOfObjectOrientedDesign
  • You can write maintainable and reliable procedural code?

2.3What is software engineering?

  • Engineering is the application of science.
  • Civil engineering is the application of natural science.
  • Software engineering is the application of computer science.
  • What is science?
    • Science is the application of the scientific method.
    • Science is a mixture of philosophy, mathematics, and experiments.
  • What is software?
    • chapter, "What is software?: The role of empirical methods in answering the question", description, pdf
      • "Legislation is (like?) software development."
      • "[Cooking] Recipes are software."
    • The ideal software is easy to change but doesn't change. The ideal software captures the essence of the problem. The essence of a problem is mathematical definitions. Mathematical definitions aren't supposed to change.
    • Software is a model of reality.
    • Software is law? Law is software?
      • Similarity between software and law
        • Bad software and bad law both result in unnecessarily slow system.
        • Both are based on logic.
        • Both have an ontology.
        • The law of a country is a big (possibly inconsistent) logic program.
          • The law in writing vs the law in practice
            • I think it is too much for anyone to know all laws that are in effect in a country.
      • Difference between software and law
        • A judge cares about both the letter of the law and the spirit of the law.
        • A computer cares only about the letter of the software. There is no such thing as the spirit of the software.
          • A computer does what we write, not what we mean.
            • The programmer follows the computer's rules. Not the other way around.
        • WP:Letter and spirit of the law
    • Software is executable theory of nature.
    • Software is like physics but executable.
    • https://queue.acm.org/detail.cfm?id=2693160
    • https://www.cs.umn.edu/research/research_areas/software-engineering-and-programming-languages
      • "Software is a solution to a computational problem using a formal programming language."

2.4What is the difference between a programmer and a software engineer?

The difference is in the amount of complexity that they can handle, the amount of foresight they make.

A programmer does not have to be backed by science. An engineer has to be backed by science.

It is the difference between bricklayer and an architect.

Commander and general.

Programming is tactical. Engineering is strategic.

A bricklayer can build a wall, or even a house, but not a skyscraper. A programmer can build a calculator, or a game, but not an operating system.

Bricklayer is less prestigious than architect, but they require a different skill set, and building requires both. Without the architect, the building would be shit. Without the bricklayer, the building would not exist at all in the first place.

Of course there are incompetent architects who build buildings despised by tenants, as there are incompetent software engineers who build software despised by users.

But software is much more complex than bridges and skyscrapers.

What is the science of software engineering? Classical physics is the science of civil engineering, and civil engineers understand physics well enough to predict everything they need to predict. If computer science is to be the science of software engineering, software engineers don't know computer science well enough to predict everything they need to predict. Worse, what if software is not predictable? What if the complexity is essential and irreducible?

Sometimes engineering front-runs science, and the engineer's ignorance harms lives. Coal mine explosions. Leaded gasoline. Microplastics. DDT. How can we predict what people will use new technology for? How can we predict the unintended consequences?

Engineers have to combat ignorant management decisions that harm people. Do you want to have blood on your hands?

Accidents happen. Buildings fall. Programs crash. People make mistakes, mostly honest.

When I get on an airplane, I want to get on one made by engineers, not business people. I want engineers to make the final decision.

If software is a pure art, then its construction should not be time-limited. A painter does not paint with deadlines. A painting is finished when the painter feels that it is finished. However, a painter, when asked to paint a very specific painting, can estimate how many days it will take.

How far can the civil-engineering analogy stretch before it breaks?

2.5What does a programmer do?

2.5.1Slave drivers

Programmers play the same role as ancient slave drivers. They translate and impose the master's will into the slaves.

Do programmers have god complex?5

2.5.2Formalization

One job of the programmer is to formalize the end-user's implicit ontology into the corresponding formal ontology that can be manipulated by machines. The programmer's job is to formalize the end-users' mental model into a formal ontology.

Requirement gathering and analysis.

Semantic-preserving translation from user syntax (natural language) to computer syntax (programming language).

Will controlled natural languages enable everyone to program the computer?

2.5.3Programming as progressive relaxation of simplifying assumptions

Programming is progressive relaxation of simplifying assumptions.

But has that activity already has another name: modeling.

We start with the assumption that the system does nothing.

Then we relax the assumption that all state is stored in memory.

Then we relax the assumption that errors do not happen.

Then we relax the assumption that the network latency is zero.

Then we relax the assumption that the user is willing to wait forever.

And so on.

2.5.4Most "bugs" are actually "programming errors".

Programmers need to own up their mistakes, and call an error an error, not a bug. A bug is something the programmer cannot control. A cosmic ray strike flipping a bit in RAM is a bug. A typo in the source code is not a bug.

2.5.5Programming, model, and reality

  • When we write a program, we are actually creating a mathematical model of reality, creating an implicit ontology, defining what exist, making simplifying assumptions, discarding irrelevant aspects.

2.6Principles are just heuristics

Heuristics are rule-of-thumbs, not absolute truth.6

2.7What are the ICSE most influential articles?

ICSE (International Conference on Software Engineering) most influential articles

  • "ICSE is the flagship conference of [software engineering] […]." (Georgios Gousios)
    • 2008, article, "Debugging Reinvented: Asking and Answering Why and Why Not Questions About Program Behavior ICSE-30"
    • 2012, slides, "Connecting Software Architecture to Implementation: The Next 10 Years", pdf
      • 2002, article, "ArchJava: Connecting Software Architecture to Implementation", pdf
    • 1997, article, "Software processes are software too, revisited", pdf
      • 1987, article, "Software processes are software too", pdf

2.8Software engineering research, advances in software engineering?

Zeller in ICSE 2018 presentation7 points out that Begel & Zimmermann 2014 survey [1] contains questions that can stimulate software engineering research.

2.9What is the relationship between software and computation?

3Building big software

3.1How should we build software, especially big software?

For big enterprise software, we should at least use model-driven engineering.

For a throw-away shell script, just open a text editor and be done with it.

For critical software, perhaps formal methods?

Brooks & Kugler 1987 [2] have warned us about the essential irreducible complexity of software. Dijkstra 1972 [3] has warned us to control complexity with meaningful abstractions. We must understand what we do, and do what we understand. Model-driven engineering should be able to minimize the accidental complexity. The essential complexity of a software system is the minimum information required by software engineers to make the system. For example, "an online shop" is insufficient information: There are too many possibilities. What are we going to sell? How much traffic do we expect? What about the operations?

Some software is so big that we cannot finish it in two weeks. But in two weeks we are already a different person. Thus we should think of ourselves as three people: our past self, our present self, and our future self. How can these three people cooperate? If we keep feeling that our past self is stupid, if we don't trust the decision made by our past self, how can we progress at all, how can we build on something, how can we grow something? Our two-week-ago self did think that it was the best decision. If we cannot respect our past decisions, we are condemned to eternal refactoring. Software engineering, self-trust, self-peace?

The relationship between programming and past self?

The key to being a good programmer is being someone that our future self can trust.

When we change a program, we are trusting our past self. We assume that our past self did not plant time-bombs that will surprise our current assumptions.

I think software development can be sped up by minimizing the gap between the end-user's mental model and the programmer's mental model. Matching mental model is a requirement of good user experience.

3.2Why people don't adopt metaprogramming tools (and model-driven engineering?)

I fear that the tool doesn't perfectly fit my use case. When that happens, I want to drop down to the lower layer. Generated code is usually inflexible. I could maintain a patch, but isn't it (patching generated code) rather hacky and brittle? The right way is to design the metalanguage so that it can express such transformation.

https://www.eclipse.org/epsilon/doc/articles/eugenia-patching/

3.3How should we specialize programmers?

In 2019 it is common to divide programmers into frontend programmers and backend programmers. I think this is wrong. It is like slicing a cake into layers instead of sectors: One person eats all the topping, another one the butter, another one eats the flour. However, if we talk about making instead of eating a cake, then it makes sense to specialize the chefs into topping specialists and dough specialists. Thus, is software engineering more like eating cakes or making cakes?

We should divide programmers into business programmer and language programmer instead. The language programmer creates a DSL (domain-specific language). The business programmer captures user requirements into a program (a specification) in that DSL. The language programmer writes a metaprogram that translates the business programmer's declarative program into something efficient.

3.4How do we make reliable software?

Make restarts cheap, quick, and easy, like in Erlang.

Reliability is not about the uptime of a single operating system process. Reliability is about the uptime of the whole system.

There are two attitudes:

  • Strive to write one fool-proof program that does not leak resources. This may be possible for operating systems, but is too hard for user applications? Let the OS handle the corner cases?
  • Use another system component's strengths to compensate for the program's weakness. Avoid stupid leaks, but do not strive to completely preclude resource leaks in corner cases. Make restart cheap and fast, and restart the program often. If the program is a web server, put it behind a load balancer, and rotate several instances often. This is more sane, efficient, and effective.

Example insanity that arises from writing code that tries to be too smart: What to do if POSIX close fails?8

Corner cases exist in the first place due to bad design, unclear semantics, lack of foresight.

Robustness can be attained by a combination of fail-fast and restart-fast.

3.5The context of "Premature optimization"

http://wiki.c2.com/?PrematureOptimization

Knuth was telling us to focus on what matters.

The catchy maxim "premature optimization is the root of all evil" is at risk of being taken out of context.

(Straw-man) example where "optimizing" is stupid: You're building a small to-do list app, and you do it in C, and you use GOTO statements.

Example where not "optimizing" is stupid: You're building a electronic fuel injection system, and you don't know anything about jitter, latency, and real-time systems. Controlling latency here is a requirement, not an optimization.

3.6Method Wars

What is SEMAT (Software Engineering Method And Theory)[https://semat.org/]? The situation is like in the XKCD competing-standards comic? There were OOP Method Wars.910" in the pdf of that book https://link.springer.com/content/pdf/10.1007%2F978-3-319-73897-0.pdf]

Are we really doing Scrum?11

3.7How to group code? By feature? By dependency? By author?

Parnas modularity, abstraction, information hiding

3.8Distributed systems

3.9Writing software?

  • Minimize build time.
  • Minimize the time from program startup to program ready. Otherwise you won't test the program.
  • Understand which parts of the code have more risk. Risk is caused by something you don't understand. (Warren Buffett)
  • Minimize the way things can go wrong. If you make a variable immutable, there are less ways it can go wrong. (?)
  • Explicit is better than implicit? Prefer writing boilerplates to magical reflection stuff. Let the compiler help you. Let compilation error guide refactoring.
  • Minimize duplication? Minimize duplication of constants, literals, fragments?
  • Data is more important than code? The shape of the data is important?
  • Make every part understandable in isolation?
  • Avoid nulls? If you must use null, document it.

3.10Idea: Database should be a library, not a stand-alone server

3.11Object-oriented design?

Meilir Page Jones, "What every programmer should know about object-oriented design"

3.12Enterprise application and plumbing?

An enterprise application usually contains some complex logic and lots of plumbing, so we need to make plumbing trivial. All our programs read data, do something what that data, and write data. The most important things are the data and the business logic.

3.13What cause software errors?

  • Programming error: what the programmers think are not what they write. Difference between their thought and the computer's actual rules. Silent wrong assumption.
  • Contributing factors: bad languages, bad tools.
    • Bad languages make writing correct programs difficult.
  • Errors outside programmer control: cosmic rays, hardware problems

3.14Optimizing enterprise application development?

3.15How do we develop enterprise software with less effort?

3.16Ontology?

  • 2005, article, "Practical Ontology Systems for Enterprise Application", pdf
  • 2006, article, "Applications of Ontologies in Software Engineering", pdf

4Software maintenance

4.1What is software maintenance?

Software does not break down like machines. It is the things around the software that change. Hardware changes. Laws change. People change. The world changes. What was true when the software was made is no longer true.

But why do we change? Other animals have been living just fine for millions of years without much change. We change for fun, to avoid boredom.

It is fun to change things. It is less fun to be affected by changes.

The key to maintainable software is to depend only on dependable things. If A depend on B, then B should be more stable than A. If a program uses a library, then the library should be more stable than the program. If a library uses an operating system, then the operating system should be more stable than the library.

Stable dependencies principle12?

How to write a program that does not change? It must capture the timeless essence of reality. A bachelor is unmarried by definition, and that definition is unlikely to change.

4.2Why does software change?

A software system, once written, works forever: It will forever satisfy its specification.

The necessity of change originates outside the software system itself.

Examples of causes of change:

  • changes in tax laws
  • growing number of users

4.3What is architecture and maintainability?

Architecture is the set of things that are hard to change. (Who said this?)

A system is maintainable iff likely changes are easy.

A system is maintainable iff the difficulty of a change is proportional to the likelihood of that change.

A system is maintainable iff the change in implementation is proportional to the change in specification. Thus, in such system, every small change in specification translates to small change in implementation.

Maintainability is about foreseeing what changes will be likely. Maintainability, like security, cannot be an afterthought. They must be designed from the beginning.

To maximize maintainability, we must code at a level of abstraction as close as possible to the specification.

Must be weighed probabilistically. It is unlikely that we will ever want to change an accounting system to a city-building game.

Example of architectural change: Adding undo capability to a software system that was not designed with undo in mind. Adding undo requires reification of all undoable actions.

4.4Backward-compatibility complicates development but simplifies maintenance

Reasons for maintaining backward-compatibility:

  • If you maintain your library's backward-compatibility, you only need to maintain the latest version of the library.
  • Compatibility reduces dependency hell and bit rot.
  • The root cause of dependency hell / DLL hell / JAR hell is short-sighted incompetent library developers who don't care about compatibility.

Reasons for breaking backward-compatibility?

  • Backward-compatibility hampers changing the interface (including fixing interface mistakes). But you can always introduce a new interface and deprecate the old one. But you will then collect junk.

4.4.1Defining interfaces and backward-compatibility

An interface is something that users depend on.

Your library's interface is what breaks your users' code if that interface changes.

Examples of interface:

  • Function parameters.
  • Performance. Example: Replacing an algorithm with a slower one may stop your users from satisfying their non-functional requirements.

A backward-compatible change is a change that doesn't annoy the people who depend on you. Such change doesn't require those people to react. Such change doesn't break those people's works.

  1. Backward-compatibility obviates backporting.

4.4.2The key of library-writing is empathy.

Think about your users. Don't break their stuffs.

See Eternal compatibility in theory.

Making only backward-compatible changes simplifies the lives of people who depend on you.

Library authors must maintain some backward compatibility. They can't just make arbitrary changes and break things.

4.4.3Choosing a versioning policy

Use what everybody else is already using in your ecosystem: Dhall versioning policy, Haskell PVP Specification, Semantic Versioning, etc.

If everybody followed eternal compatibility, versioning policies would be irrelevant and upgrades would be smoother.

4.4.4Following Haskell package versioning policy

4.4.5Backward-compatibility hall of fame

These systems may be too backward-compatible.

2018-09-01: IBM Z mainframes. The page says that the 2018 system is mostly backwards-compatible to the 1964 system.

Windows 95: Much more than you would ever know.. The original version of Sim City was writt… | Hacker News

4.4.6Respect legacy code because it works, but don't get too attached to anything.

It may be ugly, but it works.

  • Is the code really ugly?
    • Are you refusing to read it because it doesn't fit your taste? Indent size? Snake case vs camel case?
      • Is the code truly unreadable, or are you just an asshole?
        • How hard have you tried?

4.5Forward-compatibility?

4.6Logging

4.7<2017-05-20> What helps writing maintainable software?

  • Minimize duplication? The programming language limits deduplication?
  • Facilitate change, minimize ripple due to change
  • Flexibility of a component is proportional to its risk of changing?

4.8How do we know that a program is maintainable?

Change all the programmers. If the new programmers can handle it, it's maintainable.

The only way to ensure software maintainability is periodic rotation of programmers? We should rotate programmers, no less frequent than every year, for software maintainability.13

4.9How do we divide labor? How do we work together?

Interfaces enable us to pretend that our coworkers have finished their jobs.

Interfaces are contracts, promises, agreements.

4.10Hotspots and software maintenance?

Adam Tornhill: "A Hotspot is complicated code that you need to work with often."14 Our eagerness to refactor a piece of code should be proportional to how likely it is to change. We should not refactor code that will never change.

5Doesn't belong here?

Most of these things are too tactical, not strategic enough.

5.1How do we reverse-engineer software?

I want something like Ghidra15 but written in Prolog and for PS1 MIPS.

Can we use Ghidra for reverse-engineering PS1 games?

5.2Ramblings? Wish lists? Too speculative? Unclear goal/reason? Condemned to oblivion?

5.2.1Probably irrelevant to our goal

Automatic deglobalization of C programs?

5.2.2Garbage collection

5.2.3Common Lisp advocacy?

2018, article, How the strengths of Lisp-family languages facilitate building complex and flexible bioinformatics applications | Briefings in Bioinformatics | Oxford Academic

Common Lisp Object System: A Brief Guide to CLOS

5.2.4TODO Process these informations

5.2.5Composition, composable systems

5.2.6Refactoring

5.2.7ungrouped

5.2.8Designing APIs?

An API Ontology - Literate Programming

5.3Engineering ethics

Nature is objective, cold, and consistent. The laws of nature is practically unchanging.

Nature does not care. If you make wrong calculations, your structure is going to fall and kill people.

When a politician lies, he wins, because he is lying to humans. When an engineer lies, he loses, because Nature does not care about our beliefs.

Nature does not respect beliefs.

It is easier to tell the truth than to lie. To lie, one has to think hard to perpetuate the lie. But politicians don't even lie; they don't even bother trying to make their lies consistent. To them, words incite emotion, not convey information.

5.4Machine-learning database indexing; an index is a model of the data

http://deliprao.com/archives/262

https://arxiv.org/abs/1712.01208

https://blog.codinghorror.com/why-cant-database-tables-index-themselves/

5.5What is a database that can be scaled up without downtime?

We are willing to sacrifice consistency for tracking data.

5.6Java stuff

5.7Rants

React reinvents Windows 1.0 (1985)16

5.8<2018-05-30> Enterprise application stack?

  • Java 8
  • Jetty 9.4 (Servlet API 3.1.0)
  • JPA API 1.0.2 (annotations only)
  • J2HTML
  • PostgreSQL 9.5
  • J2HTML-like for C#
  • Java: Install OpenJDK 8, install IntelliJ IDEA, you're all set.
    • IntelliJ IDEA comes with Maven.
  • .NET: Install Mono, install Monodevelop 5.
    • Monodevelop 5 comes with Nuget.

5.9Git

  • Don't use GitFlow. Use continuous integration and continuous deployment instead.
    • The master branch always builds, always runs, and is always releasable.
    • Everything goes to master. Sketches, hotfixes, everything.
    • Avoid non-master long-lived branches.
    • It is OK to introduce half-baked code into master as long as it doesn't introduce errors.
      • Use feature switch/toggle.
        • Just because code is in master doesn't mean it will be run on production.
      • Work-in-progress doesn't have to be full of errors.
    • Discipline yourself to make small commits and avoid big merges.
    • Discipline yourself to make backward-compatible changes, and deprecate things slowly.
      • Give your users time to migrate.
      • If your changes are always backward-compatible, you will never break someone else's code.
    • You can rebase/tidy up commits that you haven't pushed.
    • Merges and rebases produce the same working tree but have different uses.
      • The computer doesn't care whether you merge or rebase.
      • Merges and rebases exist for the humans.
    • 2017, article, Moving away from GitFlow

5.10Things we use, tools and services?

5.11Undigested

  • BPMN tool: camunda editor (bpmn.io).

5.12Software legality?

  • Harmony CLA (Contributor License Agreement)?

5.13Market information, compensation, salary, wage

<2019-01-15> Haskell tutelage $75/hour https://twitter.com/luqui/status/1084837990877327360

I thought about software engineer salary.

A guy is charging €639 for a two-day F# course17

5.14What

  • 2010, article, "We show how symbolic execution and Satisfiability Modulo Theories (SMT) solvers can be gainfully employed to greatly automate software debugging of evolving programs.", downloadable as pdf
    • confusing title: "Debugging as a Science, that too, when your Program is Changing"
  • Distilling the best practices and standardizing the tools and processes
  • 2014, article, "Bidirectionally tolerating inconsistency: partial transformations", pdf
    • 1990, article, "Tolerating inconsistency", Robert Balzer, pdf

5.15Comparing build systems?

  • Haskell Shake
  • F# FAKE
  • Apache Buildr

5.16dependency management?

5.17Why is it so hard to compose software?

I like Emacs Org-Mode's outline editing, and I like VSCode's Search (Ctrl+Shift+F), but why can I not have a single application that has both of them?

Operating systems are a way to compose software.

5.18Model-driven software engineering?

Eclipse Epsilon18 looks interesting, but Java makes me have second thoughts.

5.19Open-sourcing

The benefit of open-sourcing for your company, if done right, is that it reduces maintenance cost, because it outsources much of the maintenance to the community. This assumes that you have a community.

6Measure productivity in software development?

22 people's ideas about measuring software development productivity:

A specification is a logical formula in conjunctive normal form. (Discussion: should it be in prenex normal form or Skolem normal form instead?)

Each formula has an essential complexity that is inherent and irreducible.

A shortest specification of a program has the least number of clauses.

To develop a program is to gradually satisfy the clauses in a shortest specification.

A programmer's productivity is the number of clauses in a shortest specification he can satisfy per unit time.

A programmer's productivity is the number of WBS (work breakdown structure) items he can satisfy per unit time.

Surface (syntactic/formal/accidental) productivity and deep (semantic/meaningful/essential) productivity.

7Bibliography

bibliography:default

[1] Begel, A. and Zimmermann, T. 2014. Analyze this! 145 questions for data scientists in software engineering. Proceedings of the 36th international conference on software engineering (2014), 12–23. url: <https://homepages.dcc.ufmg.br/~figueiredo/disciplinas/papers/icse14begel.pdf>.

[2] Brooks, F. and Kugler, H. 1987. No silver bullet. April. url: <http://www.cs.nott.ac.uk/~pszcah/G51ISS/Documents/NoSilverBullet.html>.

[3] Dijkstra, E.W. 1972. The humble programmer. Commun. ACM. 15, 10 (1972), 859–866. url: <https://www.cs.utexas.edu/~EWD/transcriptions/EWD03xx/EWD340.html>.


  1. https://www.etymonline.com/search?q=science

  2. https://en.wikipedia.org/wiki/Falsifiability

  3. https://www.quora.com/Do-you-use-SWEBOK-Is-it-helpful

  4. https://martinfowler.com/bliki/Swebok.html

  5. https://www.reddit.com/r/java/comments/2jona4/why_does_everyone_hate_eclipse/clea9ok/

  6. https://programmingisterrible.com/post/176657481103/repeat-yourself-do-more-than-one-thing-and

  7. https://www.youtube.com/watch?v=U5jLjcxnwfU&list=FLVLsoyeokV21f0G7Z39gEUQ

  8. https://stackoverflow.com/questions/33114152/what-to-do-if-a-posix-close-call-fails

  9. description of 2018 book "The essence of software engineering" https://link.springer.com/book/10.1007/978-3-319-73897-0

  10. see the chapter "Escaping method prison […

  11. http://www.allaboutagile.com/the-scrumbutt-test/

  12. http://wiki.c2.com/?StableDependenciesPrinciple

  13. https://www.yegor256.com/2015/12/29/turnover-is-good-for-maintainability.html

  14. https://www.empear.com/blog/parse-a-monster/

  15. https://www.youtube.com/watch?v=Sv8yu12y5zM

  16. https://bitquabit.com/post/the-more-things-change/

  17. https://www.avanscoperta.it/en/training/practical-machine-learning-with-functional-programming-workshop/

  18. https://www.eclipse.org/epsilon/