Our Blog

Visualizing Data with Pair-Plot Using Matplotlib

By Kürşat Kutlu Aydemir
April 25, 2022

Photo by Sebastian

Pair Plot

A pair plot is plotting “pairwise relationships in a dataset” (seaborn.pairplot). A few well-known visualization modules for Python are widely used by data scientists and analysts: Matplotlib and Seaborn. There are many others as well but these are de facto standards. In the sense of level we can consider Matplotlib as the more primitive library and Seaborn builds upon Matplotlib and “provides a high-level interface for drawing attractive and informative statistical graphics” (Seaborn project).

Seaborn’s higher-level pre-built plot functions give us good features. Pair plot is one of them. With Matplotlib you can plot many plot types like line, scatter, bar, histograms, and so on. Pair-plot is a plotting model rather than a plot type individually. Here is a pair-plot example depicted on the Seaborn site:

Using a pair-plot we aim to visualize the correlation of each feature pair in a dataset against the class distribution. The diagonal of the pairplot is different than the other pairwise plots as you see above. That is because the diagonal plots are rendering for the same feature pairs. So we wouldn’t need to plot …

Perl Web Frameworks

By Marco Pessotto
April 19, 2022

CGI

When I started programming, back in the day, CGI (the Common Gateway Interface) was still widely used. Usually the Apache webserver would just execute a script or a binary with some environment variables set and serve whatever the executable sent to the standard output, while keeping the standard error in the logs.

This simple and straightforward mechanism can still be used for small programs, but larger applications usually want to save the start-up time and live longer than just a single request.

At that time Perl was used far more often than now, and it had (and still has) the CGI.pm module to help the programmer to get the job done.

#!/usr/bin/env perl

use utf8;
use strict;
use warnings;
use CGI;

my $q = CGI->new;
print $q->header;
my $name = 'Marco';
print $q->p("Hello $name");
print "\n";

And it will output:

./cgi.pl
Content-Type: text/html; charset=ISO-8859-1

<p>Hello Marco</p>

Here the script mixes logic and formatting and the encoding it produces by default tells us that this comes from another age. But if you want something which is seldom used and gets executed on demand without persisting in the machine’s memory, …

Quartz Scheduler as a Service

By Kürşat Kutlu Aydemir
April 18, 2022

Photo by Mat Brown from Pexels

Quartz Job Scheduler

“Quartz is a richly featured, open source job scheduling library that can be integrated within virtually any Java application — from the smallest stand-alone application to the largest e-commerce system.” (Quartz Scheduler overview)

Besides its advanced features, most basic and frequently used feature is job scheduling and job execution. Some frameworks like Spring Scheduler have their integration practice using Quartz Scheduler which allows using its default scheduling method.

In this post I am going to tell you a different approach to show how we can use Quartz Scheduler to schedule our jobs. We actually still will be using the existing scheduling mechanism of Quartz but we’re going to show how we can manage the scheduled and unscheduled jobs online. This way you can manage all the available jobs or create new ones on the fly.

Quartz Scheduler as a Service

Previously I led development of an enterprise “Business Service Management” software to replace IBM’s TBSM product at a major telco company in Turkey. This was a challenging project and found a solid place in the customer environment after …

Job opening: VisionPort support engineer (western U.S.)

By Alejandro Ramon
April 14, 2022

This position has been filled. See our active job listings here.

We are looking for an engineer to join the End Point Immersive and Geospatial Support (I+G) Team—​a small, multidisciplinary team that supports our company’s clients with their VisionPort systems incorporating Liquid Galaxy technology. VisionPort hardware consists of large-panel HD TVs within a curved panoramic environment, supported by a server stack with power, video, audio, and network connections and equipment.

The candidate will be based out of a home office in the western United States (Washington, Oregon, California, Idaho, Utah, Nevada, Arizona, Montana, Wyoming, New Mexico, and Texas). The engineer will be asked to travel to, perform, and supervise system installations, in addition to day-to-day remote support work from a home office.

Occasional evenings and weekend on-call shifts are shared amongst the team.

This is a great entry-level opportunity for people already familiar with light audiovisual, server room, and/or installation handiwork to get experience with all aspects of production computer systems and their deployment. More experienced individuals will have the opportunity to work directly in feature development on production systems and possibly assist with …

On Shapefiles and PostGIS

By Josh Tolley
April 2, 2022

Partial map of the voyage of the Endurance, from “South”, by Ernest Shackleton

The shapefile format is commonly used in geospatial vector data interchange, but as it’s managed by a commercial entity, Esri, and as GIS is a fairly specialized field, and perhaps because the format specification is only “mostly open”, these files can sometimes be confusing to the newcomer. Perhaps these notes can help clarify things.

Though the name “shapefile” would suggest a single file in filesystem parlance, a shapefile requires at least three different files, including filename extensions .shp, .shx, and .dbf, stored in the same directory, and the term “shapefile” often refers to that directory, or to an archive such as a zipfile or tarball containing that directory.

QGIS

QGIS is an open-source package to create, view, and process GIS data. One good first step with any shapefile, or indeed any GIS data, is often to take a look at it. Simply tell QGIS to open the shapefile directory. It may help to add other layers, such as one of the world map layers QGIS provides by default, to see the shapefile data in context.

GDAL

Though QGIS can convert …

Extending Your Jetty Distribution’s Capabilities

By Kürşat Kutlu Aydemir
March 31, 2022

What is Jetty?

“Jetty is a lightweight highly scalable Java-based web server and servlet engine.” (Jetty Project)

Jetty can run standalone or embedded in a Java application and the details about running a Jetty webserver can be found in the Jetty Project Git repository and documentation as well. The Jetty project has been hosted at the Eclipse Foundation since 2009 (Jetty, Eclipse).

Know Your Jetty

In many legacy environments using the Jetty web server there may be an older version of Jetty. If you know the version of the Jetty distribution in your environment then you can find its source code in the Jetty project GitHub repo. Some of the distributions are in project releases but most of the distributions can be found in the tags as well.

For instance jetty-9.4.15.v20190215 distribution can be found in the Jetty project tags at this URL: https://github.com/eclipse/jetty.project/releases/tag/jetty-9.4.15.v20190215

When you clone the jetty.project Git repo, you can then easily switch to any specific release tag:

$ git clone git@github.com:eclipse/jetty.project.git
$ git checkout jetty-9.4.15.v20190215

Then you can build or add your custom code in that version.

Extending …

Working around SPF problems delivering to Gmail

By Jon Jensen
March 30, 2022

Photo by Garrett Skinner

Email deliverability

Legitimate email delivery keeps getting harder. Spammers and phishers never stop flooding everyone’s inboxes with unwanted and harmful email, so automated defenses against junk mail are necessary. But they are not perfect, and good email sometimes gets flagged as spam.

When sending important “transactional” email such as for account confirmations, password resets, and ecommerce receipts, it is often worth using a paid email delivery service to increase deliverability. Those typically cost a flat amount per month for up to a certain quota of outgoing email, with overage charges for messages beyond that.

Many of our clients use one of those services and generally they have all worked well and differ mostly in pricing and feature set. Popular choices include SendGrid, Mandrill, Postmark, Mailgun, and Amazon SES.

We continue to have many cases where we want to be able to send potentially large amounts of automated email to ourselves, our clients, or our systems. This is usually for testing, notifications, or internal delivery to special mailboxes separate from our main mailboxes.

These other uses for sending email keep us …

Code Reviews

By Kevin Campusano
March 21, 2022

Last week, a few End Point team members and I came together to prepare a presentation on code reviews for the whole company. We went through the basics of “what”, “why”, and “how”.

We also, and perhaps most interestingly, made several recommendations that we’ve discovered after years of doing code reviews in a variety of teams and project sizes. A series of “lessons learned” so to speak.

I thought it’d be useful to capture that discussion in written form. Let’s start with the basics.

What is a code review?

Wikipedia’s article on code reviews says that a code review is…

A software quality assurance activity in which one or several people check a program mainly by viewing and reading parts of its source code, and they do so after implementation or as an interruption of implementation.

That is a precise but frankly wordy way to say “having somebody look at the code you’ve written”. This definition, however, touches on a few aspects that give us good insight into what code reviews are and what their purpose is.

First up, it tells us that code reviews are a software quality assurance …