Our Blog

Job opening: PostgreSQL database administrator & developer

By Jon Jensen
November 13, 2020

This position has been filled. See our active job listings here.

Photo by manhhai, cropped & filtered, CC BY 2.0

We are seeking a full-time PostgreSQL database administrator and developer to support and consult with our clients. For this role we prefer those living in the Western Hemisphere.

End Point is an Internet technology consulting company based in New York City, with 50 employees serving many clients ranging from small family businesses to large corporations. The company turned 25 years old this year!

Even before the pandemic most of us worked remotely from home offices. We collaborate using SSH, Git, project tracking tools, Zulip chat, video conferencing, and of course email and phones.

What you bring:

5+ years of hands-on professional experience and technical expertise with:

• Database design and systems architecture
• Advanced SQL and query optimization
• Programming for web applications, stored procedures, and scripting, in Ruby, Java, Python, PHP, Perl, JavaScript, C#, Go, PL/pgSQL, bash, etc.
• Performance tuning, scalability, connection pooling
• Cloud database hosting services, especially at AWS, Google Cloud, and Azure
• Security and encryption
• Replication, PITR, backup, monitoring, and relevant tools
• HA, failover, and disaster …

Rapid Test-Driven Development in Julia

By Kamil Ciemniewski
November 9, 2020

[//]: # ( Kamil wrote: image I’ve obtained from www.freepik.com where I have a paid account. The license type says: Premium license (Unlimited use without attribution). https://www.freepik.com/free-vector/isometric-automated-production-line-concept-with-industrial-conveyor-belt-robotic-mechanical-arms-isolated_10055534.htm )

The Julia programming language has been rising in the ranks among the science-oriented programming languages lately. It has proven to be revolutionary in many ways. I’ve been watching its development for years now. It’s one of the most innovative of all the modern programming languages.

Julia’s design seems to be driven by two goals: to appeal to the scientific community and to achieve the best performance possible. This is an attempt to solve the “two languages problem” where data analysis and model building is performed using a slower interpreted language (like R or Python) while performance-critical parts are written in a faster language like C or C++.

The type-system is what allows Julia to meet its goals. The mix of strong and dynamic typing enables Python-like productivity with C++ or Rust-like performance. Julia is not an interpreted language. It …

.NET 5 will be released at .NET Conf 2020

By Juan Pablo Ventoso
November 4, 2020

.NET — A unified platform by Microsoft

Last year, at .NET Conf 2019, Microsoft announced that the new .NET 5 will have a base class library that will allow creation of any type of application for any platform — Windows, Linux, Android, iOS, and IoT. And that’s finally about to happen: .NET 5 will be launched at .NET Conf 2020, starting November 10th!

According to Microsoft, .NET 5 will take the best of .NET Core, .NET Framework, Xamarin, and Mono and merge them into one framework, offering the same experience for all developers, regardless of the type of application or platform targeted. It will also include two different compiler models: just-in-time (JIT, prepared for client-server and desktop apps) and static compilation with ahead-of-time (AOT, optimized to decrease startup times, ideal for mobile and IoT devices).

Some of the key features will be:

Windows Desktop development (WPF/​Windows Forms/​UWP)

This was already a part of the current .NET Core 3 release, and it will stay while getting some updates like a Chromium-based WebView control, improvements to the visual designer, and customizable task dialogs. It will also include all the latest features of C# 8.

Full-stack web …

Craft: A CMS for developers

By Kevin Campusano
October 31, 2020

As a software engineer, I thrive and thoroughly enjoy working on fully custom software products, applications conceived to model and help in the execution of some business process and that are built from the ground up by a team of developers.

Such projects are often complex and expensive, though, and for some clients, they can be overkill. Some clients come up with requirements that are better served by off-the-shelf software solutions. One group of such solutions are content management systems (CMS). As a rule of thumb, if a client wants a website whose main purpose is to showcase some content, their brand or image, and custom business logic requirements are limited, then chances are that a CMS will fit the bill nicely.

Lately we’ve been using the Craft CMS for a client that meets the aforementioned criteria, and I gotta say, I’ve been pleasantly surprised by the developer experience it offers.

Unlike most of the technology and products we discuss in our blog, Craft CMS is not Open Source or Free Software. The source code is readily available in GitHub for anybody to use, study, and modify, but commercial use of it is restricted and certain features are exclusive to a so-called …

EMSA: Electronic Messaging Staging Area

By Elizabeth Garrett Christensen
October 30, 2020

Photo by Lee Roberts, CC BY-SA 2.0

End Point has been involved with developing disease surveillance software for over a decade. One component of that work is EMSA, short for “Electronic Messaging Staging Area”.

The EMSA system we’re currently using was developed by a consortium of states led by Utah, Kansas, and Nevada. It is a PHP web application and can be hosted in the cloud or on-premises as you would any other PHP application.

EMSA:

• Is an ingest system for HL7 or CSV-formatted lab reports from doctor offices, labs, and other official reporting groups.
• Allows exporting XML output to your disease surveillance system.
• Is a web console for managing messages.
• Allows you to see messages that are malformed.
• Has configurable rules for disease types and message routing; some messages may go directly into a surveillance system while others may be stored for later.
• Has configurable “automated workflows” to validate message information and perform tasks.
• Can be connected to a BI (Business Intelligence) tool for reporting, or the database can be queried directly.
• Works with integration packages like Rhapsody to bring in data from other systems.

End Point has also made custom …

Detecting gaps in time-series data in PostgreSQL

By David Christensen
October 26, 2020

Photo by Phoebe Baker

A client has a number of data feeds that are supposed to update at regular intervals. Like most things in the universe (thanks, entropy!) this does not always end up working out the way we want. We recently discovered that some of the data had not loaded as expected, and were brought in to assess the extent of the issue.

There were 2 main feeds that had issues, with different types of time data. The first had date-based data with batches and the second had full-range timestamps. We will examine each type individually, since they have similar — but not the same — characteristics.

Date-based, with batches

In the first data feed there was a table which tracked which files had been loaded by file date and batch number (there was a morning/​evening batch designated by a batch field with either A or B). Since the files that had not loaded successfully did not have entries in this table we could find that one or both of the day’s batch files would be missing, but since the table tracked those which did get loaded, we needed to turn this list into something useful.

Any time I find myself considering sequences of data, I remember my friend generate_series(). While …

ROS 2 Migration

By Matt Vollrath
October 23, 2020

Migrating from ROS 1 to ROS 2 is much more than a “flip of a switch”. The internals of ROS are fundamentally changed, client libraries and their interfaces have been rewritten. The benefits of these changes are significant for users embedding ROS in a product.

Some of these benefits include:

• First-class support for real-time and embedded platforms such as microcontrollers
• Higher reliability under less-than-ideal network conditions
• Distributed, cooperative robotics systems without a single “master”
• More prescribed patterns for building systems with ROS

Because official ROS 1 support ends in 2025, the clock is ticking for all ROS 1 users to migrate their projects to ROS 2. While the amount of work required to port a project scales with its size, codebases of all sizes should have a migration plan.

By carefully approaching your ROS 2 migration, disruption to ongoing development and maintenance of the project can be minimized while maintaining the mission critical reliability required in the robotics domain.

Dependencies

If your project depends on any of the numerous free open source ROS packages, they will need to be ported to ROS 2. Much progress has been made here, but it is not …

The Pragmatic Programmer book, 20th anniversary edition

By Jon Jensen
October 16, 2020

The Pragmatic Programmer is a now-classic book about software development, first published in 1999. It is old enough that it predates the famous Agile Manifesto of 2001. The authors of the book, Andy and Dave, were two of the 14 authors of that manifesto.

For its 20th anniversary in 2019, Dave and Andy created a new edition. They updated things that had become dated, such as mentions of programming languages, tools, operating systems, websites, etc. That was, I imagine, the easy part. They went on to extensively revise the entire text and incorporate new lessons learned in the past two decades.

A classic

This book is part of our company’s “work philosophy canon” that I ask every software developer at End Point to read, so for that reason and others I wanted to be familiar with the new edition and make sure it is still something I want to recommend so broadly.

The book is also required reading for university courses at Cornell (CS 3110: Data Structures and Functional Programming) and the University of Washington (CSE 331: Software Design and Implementation), probably among others!

I first read this book 19 years ago and really enjoyed it. I found that it clearly expressed many …