Our Blog

Rclone: upload to the cloud from your command line and much more

By Ardyn Majere
September 9, 2020

The Swiss army knife of storage

Cloud storage providers like Google Drive are great solutions for storing files. You can upload your data and not worry about maintaining a separate system to host it, or all the security hassles that can bring. However, very few major cloud storage providers offer a command line interface or any other official way to upload without using their web interface or closed-source binary tools, if they even offer that.

This obviously makes uploading files from servers difficult, but not impossible if you know the right tools.

About a year ago Jon Jensen penned a blog post about gdrive, a Google Drive command-line tool. However, due to changes with Google’s Drive security, that tool no longer works. This led me to look for a replacement.

Our use case

Recently I had to put some large files in to long term storage on Google Drive, since we needed the local space back. We wanted to retain the data, but didn’t foresee needing to access it for some time, if ever. Google Drive was a good solution for us, but the problem became how to get it there.

The files were too big, and some of them were not stored sparsely—​empty space was tacked on to the disk images. We …

Containerizing Magento with Docker Compose: Elasticsearch, MySQL and Magento

By Kevin Campusano
August 27, 2020

Magento is a complex piece of software, and as such, we need all the help we can get when it comes to developing customizations for it. A fully featured local development environment can do just that, but these can often times be very complex as well. It’d be nice to have some way to completely capture all the setup for such an environment and be able to get it all up and running quickly, repeatably… even with a single command. Well, Docker containers can help with that. And they can be easily provisioned with the Docker Compose tool.

In this post, we’re going to go in depth into how to fully containerize a Magento 2.4 installation for development, complete with its other dependencies Elasticsearch and MySQL. By the end of it, we’ll have a single command that sets up all the infrastructure needed to install and run Magento, and develop for it. Let’s get started.

Magento 2.4 application components

The first thing that we need to know is what the actual components of a Magento application are. Starting with 2.4, Magento requires access to an Elasticsearch service to power catalog searches. Other than that, we have the usual suspects for typical PHP applications. Here’s what …

Our Vue Storefront “Proof of Concept” Experience

By Kürşat Kutlu Aydemir
August 10, 2020

Recently we experimented internally with integrating Vue Storefront and Magento 2.3. Vue Storefront is an open source Progressive Web App (PWA) that aims to work with many ecommerce platforms.

What initially piqued our interest was the possibility of integrating Vue Storefront with the venerable ecommerce back-end platform Interchange, which many of our clients use. Vue Storefront’s promise of ease of integration with any ecommerce backend made us curious to see whether it would make a good modern front-end for Interchange.

Since Vue Storefront seems to be most commonly used with Magento, we decided to start our experiment with a standard Vue Storefront/​Magento 2.3 proof-of-concept integration.

PoC of Vue Storefront/​Magento 2.3

OK, to be honest, at the beginning we blindly expected that Vue Storefront would be a copy/​paste front-end template solution that would fairly easily be made to work with its standard integration to a Magento backend. Sadly, this was not the case for us.

Before beginning our journey here, to summarize the Vue Storefront integration with Magento let’s have a look at this diagram to see what components are included:

Figure 1

At first, we wanted to see how …

Downtown San Diego Liquid Galaxy

By Dave Jenkins
July 27, 2020

We just installed a new Liquid Galaxy system for the Downtown San Diego Partnership in the conference room of their office in downtown San Diego (heh). As End Point continues to partner with public organizations, associations, and government agencies, the Liquid Galaxy is proving very effective for showing infrastructure projects, zoning districts, and, most importantly, public engagement with immersive data models. Downtown San Diego wanted to bring presentations and visualizations to a much larger canvas, and the Liquid Galaxy fit well with their open floor plan and large conference room.

Downtown San Diego is tasked with promoting the development of the downtown corridor to their members and the wider public. They can now build some great presentations to fully leverage the 7 large screens showing 3D models of new developments, zoning maps superimposed directly on Google Earth, and with the 4K videos all programmed to show in sequenced scenes, or simply fly through the city with a 6-axis controller and iPad.

This installation presented some unique challenges. The first was an asymmetric wall layout with a large flat wall, smaller angled wall, and an alcove that needed …

Job opening: Windows Systems Integrator

By Jon Jensen
July 23, 2020

This position has been filled. See our active job listings here.

We are looking for a Windows systems integrator in the New York City metropolitan region to work with us.

We are an Internet technology consulting company based in NYC, with 50 employees serving many clients ranging from small family businesses to large corporations. The company turns 25 years old this year!

This is a consulting position, so excellent verbal and written communication, troubleshooting, and time management skills are required, along with a good sense for when to quickly escalate issues to resolve them efficiently as needed.

Skills and tools

You will need to have extensive experience in the Microsoft Windows ecosystem: the MS Windows OS, Windows networking, Active Directory management via Group Policies, MS Exchange Server, MS SQL Server, etc.

The greater knowledge of and larger base of experience you have with these, the better:

• Remote management & monitoring (RMM) systems, such as ConnectWise Manage and Automate
• Mobile devices and mobile device management (MDM) systems, such as SOTI MobiControl, AirWatch, and MaaS360
• VMware’s vSphere or Microsoft Hyper-V hypervisors
• Managing firewall security policies, switches, and wireless access points (WAPs)
• Storage …

Deploying (Minecraft) Servers Automatically with Terraform

By Zed Jensen
July 16, 2020

Last year I bought an old Dell Optiplex on eBay to use as a dedicated Minecraft server for my friends and me. It worked well for a while, but when my university switched to online classes and I moved home, I left it at my college apartment and was unable to fix it (or retrieve our world save) when it failed for some reason. I still wanted to play Minecraft with friends, though, so I had to figure out a solution in the meantime.

I’d previously used a basic DigitalOcean droplet as a Minecraft server, but that had suffered with lag issues, especially with more than two or three people logged in. Their $5 tier of virtual machine provides 1GB of RAM and 1 CPU core, so it shouldn’t be too much of a surprise that it struggled with a Minecraft server. However, more performant virtual machines cost a lot more, and I wanted to keep my solution as cheap as possible.

I mentioned this to a co-worker and he pointed out that most companies don’t actually charge for virtual machines on a monthly basis; in reality, it’s an hourly rate based on when your virtual machine instance actually exists. So, he suggested I create a virtual machine and start my Minecraft server every time I wanted to play, …

Random Strings and Integers That Actually Aren’t

By Josh Williams
July 2, 2020

Image from Flickr user fsse8info

Recently the topic of generating random-looking coupon codes and other strings came up on internal chat. My go-to for something like that is always this solution based on Feistel networks, which I didn’t think was terribly obscure. But I was surprised when nobody else seemed to recognize it, so maybe it is. In any case here’s a little illustration of the thing in action.

Feistel networks are the mathematical basis of the ciphers behind DES and other encryption algorithms. I won’t go into details (because that would suggest I fully understand it, and there are bits where I’m hazy) but ultimately it’s a somewhat simple and very fast mechanism that’s fairly effective for our uses here.

For string generation we have two parts. For the first part we take an integer, say the sequentially generated id primary key field in the database, and run it through a function that turns it into some other random-looking integer. Our implementation of the function has an interesting property: If you take that random-looking integer and run it back through the same function, we get the original integer back out. In other words…

cipher(cipher(n)) == n

…for any integer …

Improving max() performance in PostgreSQL: GROUP BY vs. CTE

By David Christensen
June 30, 2020

Photo by Maxpax, used under CC BY-SA 2.0, cropped from original.

When working with large tables, even simple actions can have high costs to complete. What queries are acceptable for smaller tables can often be less than ideal when applied to large tables, so your specific choice of approach to a given problem becomes more important.

Note: We are using PostgreSQL 12, which supports some nice features like parallel btree index building, which can speed up parts of this process compared to earlier versions. We are using the default settings for this, which lets PostgreSQL use up to 2 parallel backend workers to speed up some operations.

Say you have a table table_a with multiple grouping fields field_a and field_b and you want to find the maximum value of another field field_c for each group.

The direct approach is to do something like the following:

SELECT field_a, field_b, max(field_c) FROM table_a GROUP BY 1,2;

This is functional and very straightforward. However, even if you have an index on (field_a, field_b, field_c), this can end up taking quite a long time if the tables are large. Let’s look at an actual example and the numbers we use.

First, let’s create our table:

CREATE …