Comparing YottaDB Web Framework Performance
Raman Sailopal
We welcome Ram Sailopal back for his second guest blog post. Read his first post on the YottaDB Dashboard here.
Background and Motivation
With its “many languages, one database” philosophy, the daemonless, language-agnostic database YottaDB, provides in-process APIs from multiple languages — C, C++, Go, Lua, M, Node.js, Perl, Python, and Rust today. Some language APIs (“wrappers”) are developed by YottaDB, and some by others in its ecosystem. While every language has its philes and phobes, there are entirely rational reasons to select a language, including:
- Developers & Tool Chains – If you have developers skilled in a language or an available proven tool chain in a language, it maybe simpler to use that language than to train your developers in another language, or to implement a new tool chain.
- Performance & Throughput – Users need to get data quickly, and as the number of users ramps up, any performance degradation should be gradual.
- Robustness – Since it is anathema to deliver incorrect results, all language stacks return correct results. However some stacks may experience drop-outs, lost connections, and timeouts, which detract from the user experience.
The basic model of a web stack consists of a web server and a back-end.
It is interesting to compare the performance of different web stacks and frameworks under simulated stress. To compare apples-to-apples, the database, the JSON string response to a REST query, and front-end load generator were the same.
Of course, this end-to-end test only involves a single operation. Any real application consists of many operations at different layers in the framework, only a fraction of which are database accesses.
Test Setup
The database consisted of the following hierarchical key-value tuples:
^PATIENTS(1,"address")="234,Timbucktwo Road, Fantasy City" ^PATIENTS(1,"age")=52 ^PATIENTS(1,"name")="Bob Taylor" ^PATIENTS(1,"sex")="Male"
which was delivered by the back-end in response to a REST query as the following JSON string:
[{ "id": "1", "name": "Bob Taylor", "age: "52", "sex": "Male", "address: "234,Timbucktwo Road, Fantasy City" }]
Each test stack used two containers both hosted on Gitpod. The front-end load was generated by Locust, running in a container with the web server and the back-end running in another.
Note: In some cases, where the same software is capable of being both the web server and the back-end, separate copies were run, in order to keep the comparison apples-to-apples. In specific situations, such as an in-house application, it may of course be appropriate to run a single copy of the software as both the web server and the back-end.
Apache is used as the web server, including for mgweb-server, which incorporates Apache. The table below lists the back-end that was used for each language. Each test container has a name for identification, and the names link to the detailed results for that stack. Native C and the Perl wrapper were not tested.
| Language | Test Container | Web Server | Back-end |
|---|---|---|---|
| C++ | yottac | Apache | Crow+Informatik Aalen CPP-Access-for-YottaDB |
| Go | yottago | Apache | Gin+YottaDB Go Wrapper |
| Lua | yottalua | Apache | Pegasus+YottaDB Lua Wrapper |
| M | yottamgweb | mgweb-server | |
| M | yottaweb | Apache | M-Web-Server |
| Node.js | nodem | Apache | Express+Nodem |
| Node.js | mgdbx | Apache | mg-dbx |
| Python | yottapython | Apache | Flask + YottaDB Python Wrapper |
| Python | yottamg_python | Apache | mg_python |
| Rust | yottarust | Apache | Rocket+YottaDB Rust Wrapper |
In each case, the load simulated 1,000 concurrent users, and each test involved approximately 100,000 requests.
As expected, running a stress test against a multi-layer application running in one container generated errors, characterized as follows:
- A web server starts servicing a request, runs out of resources, and is unable to complete the request.
- A web server lacking the resources to accept a resource for servicing.
- A crash while servicing a request.
The detailed results for each stack include the errors for that stack.
Results
Click here — Overall Results — to view the overall results, summarized below.
The surprising result is that the winners are the Lua and Rust frameworks, both delivering better performance as well as fewer errors than the native M frameworks. The C++ stack is interesting, because while it delivers good performance, its failures make it an outlier.
All tests are available at YottaWeb-Test. Please do try the tests in your environment, and if you have enhancements to add or results to submit, please submit a Pull Request on GitHub.
About Raman Sailopal
Images Used:
[1] World wide web. Photo by Svilen.milev.
[2] BCL – spiderweb. Photo by vastateparksstaff.
Screenshot courtesy Raman Sailopal.
Published on June 23, 2022