• 1,000 simulated office users worked on LowestCode for 30 minutes at full load, one action every 15 seconds each.
• Server and database together kept 1.2 processor cores busy. They ran on one device, an Apple M1 Pro, which also ran the load generator.
• In one run the users performed about 141,000 actions with 172,000 requests. Every answer was checked for its content, about 171,000 checks, and in the three runs after we corrected the test data none failed.
• Half of all requests were answered within 9 milliseconds, 95 % within 136 milliseconds.
• The processor drew about 5 watts for the whole test, load generator included. The device did not get warm.

Five runs count. A sixth was aborted and is not counted.

The test runs the office work of our insurance application in a test environment with synthetic data: 1,000 users, 16,000 customers and 16,000 offers. No customer data is involved.

Each simulated user does what an office user does. The user logs in once, which loads the workplace page and the offer list. Then the user repeats: think for about 15 seconds, then do one of seven things.

The think time varies between 10.5 and 19.5 seconds. The choice of action and offer comes from a generator with a fixed seed, so two runs replay the same sequence.

• One device, an Apple M1 Pro with 10 cores and 32 GB of memory.
• On it: the LowestCode server, the PostgreSQL 14 database with its default settings (128 MB shared buffers), and the load generator k6.
• The database held 811 MB.

The load generator ran on the same machine, so its work is part of the measured machine total. A server in a data center was not part of this test.

One run takes 44 minutes.

Three words are easy to mix up, so here is what each means.

• An action is one thing a user does: open an offer, save a row, calculate a tariff. About 141,000 per run, 58 per second.
• A request is one call to the server. Most actions send one, two of them send two, and the login sends four. About 172,000 per run, 70 per second.
• A check is our test of one answer. The answer must arrive without error and contain what was asked for: the rows of the requested offer, the saved row, the right customer in the tariff document. An answer that arrives but carries the wrong content counts as a failure. Every request is checked except the first load of the public page, about 171,000 checks per run.

Rows read by the database are a fourth number, and a much larger one. It is not a count of operations; it is explained below.

The failed checks of runs 1 and 2 were all tariff documents that lacked the customer reference our check looks for. The cause was a defect in the generated test data, not in the application: 149 of the 16,000 test customers carried a search key in a form the application itself never writes. We corrected the test data generator before run 3. No request failed in any run, and the server logged no error.

Runs 1 and 2 were made at the same state to see how well a run repeats: every figure agreed within 2 %. Runs 3 and 4 repeat within 1 % on the measured processes.

Over all requests of a run:

Per action, run 4:

The workplace page is the whole workplace with its preloaded windows, about 8.5 MB. The tariff document is about 3.2 MB.

Our own target is 95 % of all requests within 100 milliseconds. At this load on this device we measured 136 milliseconds, so the target is not met yet.

The figures are the averages of the 30 minutes at full load, sampled every 5 seconds.

• The processor power is that of the whole machine: server, database, load generator and the operating system together. The share of server and database is 2.2 to 2.7 watts.
• It is the power of the processor as the operating system reports it. It is not a measurement at the wall socket.
• Run 4 started at midnight beside the device's nightly maintenance jobs, so its machine total is higher than in run 3. The server and the database measure the same.
• Run 5 ran with a profiler attached to the server. The profiler costs processor time of its own, so the load of that run is not comparable, and no power was sampled.

That the device did not get warm is an observation of the person at the desk, and it fits the 5 watts.

Per run the database processed about 2.5 million transactions. To answer them it read rows: 1.44 billion by scanning tables and another 310 to 340 million through an index.

This number is large because it counts every row the database touches, not every answer it gives. A filter over a list of 16,000 offers touches many thousands of rows to return one page of them.

It also shows what the test is good for. In runs 1 and 2 the database read 5.45 billion rows per run. The statistics showed where: columns that link a child to its parent had no index. We made the index a rule of the structure generator, and from run 3 on the database read 1.44 billion rows and needed 0.67 instead of 1.22 cores.

Measured, on this device:

• the load of 1,000 users, their actions, requests and checked answers
• the response times
• the processor load of server and database, and the processor power of the machine

Calculated from the measurement, not measured:

• the server sizes of the comparison on our homepage, 4 to 8 cores and 20 to 31 watts per 1,000 users. They size the measured load for real operation, with headroom, a standby and a second site, on the published power data of server machines.

Not measured at all:

• any other product. The six systems of the comparison are estimated from their own published sizing rules and certified benchmarks.
• power at the wall socket, and a server in a data center
• more than 1,000 users on one machine, and the memory they would need
• the message queue service, which did not run in this test

The test data is synthetic. It proves how the system behaves, not how real customer data is distributed.

Every run left one set of files: the setup with the software revision, the result of the load generator, the processor and power samples, and the statistics of the database. The files are kept unchanged, with a checksum for each, as a versioned evidence package. The test instrument is part of the source code, so a run can be repeated.

We open the package in a technical review.