Author Archives: ksbhaskar

We wish you and yours a Merry Christmas, Happy Hannukah, Joyous Kwanzaa, Bright Karthigai – whatever you choose to celebrate this holiday season – and a Happy New Year. We hope all is well with you and yours, and that 2021 brings more sanity than 2020 did.

YottaDB moved to 100% telecommuting in mid-March, and we have been working remotely since then. We anticipate returning to the office after the pandemic; meanwhile, the office is for computers and the occasional cobweb.

Nevertheless, we have been busy this year. Our major milestone was the release of Octo 1.0, for querying YottaDB databases using SQL-92 and JDBC. Our next major target for Octo is read-write capability using SQL. Although the following is very much code that is under development, this demonstration of INSERT INTO will give you a taste of what we plan to release in 2021:

$ echo "create table usstates(state varchar primary key, capital varchar);" | octo
OCTO> create table usstates(state varchar primary key, capital varchar);
OCTO> 
$ echo "create table uspresidents (id integer primary key, firstname varchar, middlename varchar, lastname varchar)" | octo
OCTO> create table uspresidents (id integer primary key, firstname varchar, middlename varchar, lastname varchar)
OCTO> 
$ head -3 ~/.yottadb/usstates.sql
INSERT into USSTATES values
       ("Alabama",      "Montgomery"),
       ("Alaska",       "Juneau"),
$ head -3 ~/.yottadb/uspresidents.sql 
INSERT into USPRESIDENTS values
       (1,      "George", "",    "Washington"),
       (2,      "John",   "",    "Adams"),
$ octo -f ~/.yottadb/usstates.sql
$ octo -f ~/.yottadb/uspresidents.sql 
$ echo "select state, capital, id, firstname, middlename, lastname from uspresidents, usstates where lastname = state or lastname = capital;" | octo
OCTO> select state, capital, id, firstname, middlename, lastname from uspresidents, usstates where lastname = state or lastname = capital;
Washington|Olympia|1|George||Washington
Wisconsin|Madison|4|James||Madison
Mississippi|Jackson|7|Andrew||Jackson
Nebraska|Lincoln|16|Abraham||Lincoln
OCTO> 
$

Along with a number of other enhancements and fixes, in our mission to make YottaDB a language-agnostic database, we added support for Rust this year, adding to our support for C, Go and M. From our community, we have support for access to the database from Perl (thanks, Stefan Traby!), and wrappers for access from node.js (thanks, David Wicksell!) and Python (thanks, Peter Goss!) are being worked on by community members. We continue to merge upstream code releases into the YottaDB code base, while ensuring that the merged code meets the standards for robustness, performance, and security that you have come to expect from us.

Since all our software is developed as a 100% free / open source project, you can follow our work at https://gitlab.com/YottaDB. You can see what our development plans are, and you can even see our review comments on our developers’ code Merge Requests. Please do feel free to add your comments to any Issue or Merge Request.

YottaDB continues to grow. We have grown organically from the original three founders to a team of ten. In 2020, we had a table at an international event, FOSDEM 2020, in Brussels. We had more visitors there than all our events in the US in 2019 combined, but after that the pandemic shut down our in-person events. We hope to resume them in due course of time. In 2020, we also refreshed our web site, and we had additional users who permitted us to feature them on our web site.

We send you our hopes and best wishes for good health, happiness and every success in 2021, and hope that we have an opportunity to meet in person in the not too distant future.

Visiting family in Copenhagen over the holidays at the end of 2019, we were surprised to find Danes carrying packages of fireworks in local trains, which we in the United States would not dream of. Starting around 8pm on New Years Eve, there was a steady and increasing popping of fireworks, and starbursts overhead, leading to a continuous crescendo around midnight. There were fireworks in streets with pedestrian and vehicular traffic, and at one point orange volcano fireworks on the sidewalk across the street were so bright and fiery that they had us rushing to the window to investigate. Crowds gather at Rådhuspladsen, a big plaza outside the Town Hall, with the air thick with smoke. That is Danish culture. Other cultures may celebrate their new year by wearing traditional clothes.

So what is culture? According to Wikipedia, it “encompasses the social behavior and norms found in human societies, as well as the knowledge, beliefs, arts, laws, customs, capabilities and habits of the individuals in these groups.” Coming from cultures where you decline until you are pressed when your dinner host or hostess asks whether you want seconds, many an international guest at an American dinner has been dismayed to see dishes carried back to the kitchen!

Acculturation is a process in which an individual adopts, acquires and adjusts to a new cultural environment. As humans, we learn by doing, and immersion is the only road to acculturation.

What does Acculturation mean for software such as YottaDB? You can of course start with “Hello, world” examples that access the database, but what then? The function of a database is persistence: to store data entrusted to it, and provide the stored data when requested. You need to know how to install the software, configure databases to provide business continuity when systems fail or when an entire data center fails, and more.

To that end, we have created the YottaDB Acculturation Guide. It sits between the “Hello, world“ examples and the YottaDB Administration and Operations Guide, which is complete reference material that can appear overwhelming after “Hello, world”.

The one constant about culture is change! Culture is not static. The original Acculturation Guide (sometimes referred to as the Acculturation Workshop) is over ten years old, but has been significantly revamped to reflect YottaDB’s evolution as a language-agnostic NoSQL database, and simpler crash-recovery. As YottaDB gains new functionality and new operational capabilities, we will edit the Acculturation Guide from time to time, adding new material and updating existing content.

Meanwhile, please do try the latest YottaDB Acculturation Guide and tell us what you think.

Images Used:

[1] The Gumpa dance is a special dance celebrated around the time of Losar, the Tibetan New Year. Pictured here in Lachung monastery, North Sikkim, Sikkim, India. Photo by Nichalp downloaded from Wikimedia Commons.

[2] New Years Eve celebration at København Rådhuspladsen. Photo found by and downloaded from Google search. Photographer unknown – please contact us if you are the photographer or copyright holder.

[3] A village girl, Palangan, Kurdistan, Iran. Photo by Salar Arkan downloaded from Wikimedia Commons.

As YottaDBⓇ reaches the two and a half year mark, the year-end season is an opportunity for us to reflect on where we started, and where our journey has brought us.

In 2017 three of us started YottaDB, having each worked on GT.M for over twenty years. With a sense of accomplishment, we could point to our contributions. But the GT.M database would never be directly accessed by any language other than M and we believed that the database had value to applications beyond those written in M (see The Anchovy Pizza Company – Not!).

At the end of 2019, we have organically grown to a team of ten. Our office in Malvern, which felt empty when we moved in, will feel cramped when we add our next couple of team members. We anticipate moving to a bigger office when our current lease ends.

In a word, the goal of YottaDBⓇ is persistence. Any application, in any language, and on any platform, should be able to entrust its data to YottaDB. While that may be an overambitious dream, we have a roadmap.

We started by creating a native C API to the database engine. We did this in two stages, first a set of functions for single-threaded applications, and then additional functions for multi-threaded applications. As C is the lingua franca of computer languages because all programming languages have the ability to call C code and be called by it, a high-performance, native C API opened the door to YottaDB access from other languages, using “wrappers”. Our goal with each wrapper is to provide a robust, high-performance API that a programmer in that language would consider intuitive. The first language for which we created a wrapper is Go. Programming YottaDB in Go is now fully Supported with production grade code. In 2020, we hope to release wrappers for Rust and Python, with more languages to follow.

As all of our work is 100% free / open source software (FOSS), our community also contributes software to the YottaDB ecosystem. For example:

• Nodem (thanks, David Wicksell!) provides a Node.js API to YottaDB.
• A Perl pod (thanks, Stefan Traby!) provides access from Perl.
• CFMumps (thanks, John Willis!) provides Cold Fusion Markup Language access to YottaDB.
• The M Web Server (thanks, Sam Habiel and Christopher Edwards!) provides web services from YottaDB.
• QEWD (thanks, Rob Tweed!) is a Node.js-based framework for building REST APIs and interactive, real-time web applications, whilst abstracting YottaDB data as on-disk JavaScript objects.
• Chris Munt’s GitHub page (thanks, Chris Munt!) provides numerous interface packages for YottaDB, as well as his own native SQL engine.

Although YottaDB is a hierarchical key-value NoSQL database, able to handle schemas that are a superset of relational schemas, many applications have relational schemas, and even applications with non-relational schemas have relational subsets of their schemas. To accommodate the large number of tools for analytics and reporting of relational data, we are investing heavily developing OctoⓇ, a SQL engine to access relational data in YottaDB databases. Octo CREATE statements define tables and their mapping to YottaDB global variables. Octo compiles SQL queries to native object code that directly accesses YottaDB global variables with minimal overhead.

Throughout our work, we have maintained upward compatibility with upstream GT.M releases. We integrate the code of GT.M releases into the YottaDB code base. As we do this, we create automated regression tests, find, and fix bugs. Upgrading from GT.M to YottaDB is functionally 100% upward compatible, and follows the same steps as upgrading from an older GT.M version to a newer version (conversely, for M application applications, downgrading from YottaDB to GT.M follows the same steps as downgrading from a newer GT.M version to an older one).

YottaDB is a fully open project. This means that when we fix an issue or add an enhancement, under our GitLab projects you can see the code changes, you can see review comments and responses, and you can see associated automated regression tests. So when our release notes mention a fix or enhancement to the software, you have complete traceability to the associated code changes and tests. This traceability adds significant value beyond just having the code for a release available under a FOSS license.

In closing, we would like to take this opportunity to wish you a Merry Christmas, Happy Hanukah, Joyous Kwanzaa – whatever you celebrate this season – as well as a Wonderful New Year in 2020. You make YottaDB possible. We thank you and look forward to serving you in 2020.

The YottaDB Team
in alphabetic order: Bhaskar, Brad, Chris, Ganesh, Jon, Joshua, Matt, Narayanan, Saraswathi, and Steve.
with wags from: Dash and Emmy

For users of YottaDB using the Go API, r1.28 is a major release because it is required for production grade access to YottaDB from Go. For others, it is a minor release with a small set of enhancements and fixes as detailed in the complete release notes.

YottadB r1.28 is upward compatible with YottaDB r1.26, and thus upward compatible with GT.M V6.3-006 and V6.3-007.

Go Wrapper

As discussed in our blog post YottaDB Go Wrapper on February 6, 2019, we have created a package yottadB to access YottaDB from the Go language, which is among the top 10 in the IEEE Spectrum 2019 language popularity survey. Where we previously considered the Go wrapper to be field test grade, we now consider it to be production grade.

The reason that it takes a YottaDB release to make the Go wrapper production grade arises from the fact that runtime system of the most commonly used Go implementation assumes that it is in primary control of the process. As the YottaDB runtime system resides within the address space of the application process, it must protect its memory from Go. Furthermore, it must install its own signal handlers. Overcoming these hurdles, which in some cases resulted in intermittent failures that caused abnormal process termination sometimes after hours and even days of stress testing on our servers, was in equal measures art and engineering, which prolonged the release date. Our presentation Hidden Dragons of CGO discusses some of these challenges.

On a system or container with YottaDB installed, installing and using the Go wrapper is as simple as:

• Install with:

  go get lang.yottadb.com/go/yottadb

• To use, include

  import "lang.yottadb.com/go/yottadb"

  in application code

• Documentation is at https://godoc.org/lang.yottadb.com/go/yottadb (with complete documentation in the Multi-Language Programmers Guide)

We are also providing a Dockerfile including the packaged Go wrapper.

Please use the YottaDB Go wrapper, and tell us what you think!

Images Used:
Go Gopher, Renee French.
The design is licensed under the Creative Commons 3.0 Attributions license.
Read this article for more details: https://blog.golang.org/gopher