We at Leapfrog are fascinated by the sheer volume of open source projects that come across during development. These freely available open source software, with contributions from thousands of developers around the world is the cornerstone of the software industry.
2019 has been an excellent year for us, and we have been embracing open source projects like never before. With that enthusiasm, we asked the Leapfroggers to share some of the open source projects they use on a regular basis. Based on that here is a list of most loved open source programming languages, frameworks, and tools we have grown fond of and have been using in our projects.
Best Open source projects
It’s Learn Once, Write Anywhere approach makes it a perfect match for building both single page web and mobile apps. In our experience, around 70% of the code is reusable between React and React Native.
See what Nishan had to say about creating his own virtualized lists in React.
Next in line is Express. Express JS is a framework based on Node.js for building web applications and APIs. Express is our goto framework for building backend services. It is minimal and very fast to scaffold out REST APIs. The middleware architecture that is used by Express makes it very modular and extensible. From our experience, HTTPS services written in Express can be iterable without losing productivity.
We have seen our code quality improve ever since we started writing frontend and backend in TypeScript. The code is natural to read, and the debugging process is more straightforward. Another great feature about TypeScript is TSDoc that addresses these limitations of defacto code documentation JSDoc. We certainly cannot stop raving about TypeScript. The community around TypeScript is growing and, we expect TypeScript adopters to increase more in the coming year.
DBCLI is a suite of command line clients for MySQL, PostgreSQL, SQL Server, Athena and Vertica that provides auto-completion and syntax highlighting.
It provides us with a user-friendly shell alternative to the default shell provided by different databases. It has been the top pick amongst the command line interface tools at Leapfrog. Its cross-platform support makes it, even more, preferable for developers and command line tool enthusiasts.
SQLAlchemy is an open source Python SQL toolkit and object-relational mapper(ORM). It is usable with or without the ORM features. Any given project can choose to use SQLAlchemy Core or both Core and the ORM.
SQLAlchemy’s philosophy states — ‘An ORM doesn’t need to hide the “R”. It provides efficient and high-performing database access. It follows data mapper pattern and provides core python based SQL expressions. The ORM provides an open-ended set of patterns that allow a developer to construct a custom mediation layer between a domain model and a relational schema. SQLAlchemy’s modularity and extensibility make it easy for developers to go and use it’s core elements like connection pooling and transaction very trivial.
Airflow has become the tool of choice for our data engineering team after tinkering with a few workflow management platforms. Before Airflow, understanding dependencies and maintaining data pipelines was getting increasingly difficult for our data warehouses.
Luckily Airbnb had similar problems, and Airflow was created at by Maxime Beauchemin to handle their growing amounts of data pipelines. Airflow solves the issue of managing, maintaining and handling data pipelines by creating DAGs (Directed acyclic graph) from the data pipelines. Airflow can help in scheduling and monitoring workflows easily. We can define our pipelines as Python code. Across some projects, we’ve replaced the good old CRON schedule with Airflow DAGs. It has made our data warehousing workflow simple by allowing teams easily to organize and schedule them. With its powerful UI, data engineers can also easily see the progress on the various pipelines.
The QA team have voted on Robot Framework as their favorite open source tool for automation.
Robot Framework works as a generic test automation framework for acceptance testing. It also works with many testing cases/scenarios and is easy to get started. It provides you with the support for multiple testing paradigms and patterns. Being a key-driven testing framework, it uses tabular test data syntax. Written in Python, it has a good community of active contributors. Even though it is not deemed as popular, we’ve been using Robot Framework to write acceptance tests for different web applications.
If you have not used it yet, give it a try or learn how to build an extension.
Getting back to quality assurance, Locust is an easy-to-use, distributed load testing tool. Unlike Jmeter, where performance tests are created by using its GUI mode, Locust allows us to write performance scripts in pure Python. Having all our tests in code is a big advantage. It enables us to fix tests and debug it as well quickly. It is as friendly for the QA team as it is for the developers. Both the developers and the QA team can work hand in hand.
Locust is very lightweight and allows you to simulate thousands of concurrent users on a single machine easily. With some Python programming experience under you belt, you can quickly test your HTTP based applications performance in higher load scenarios.
Lastly, after a great year with open source projects, our pick ends with The ELK stack which is a collection of three open-source products — Elasticsearch, Logstash, and Kibana. The ELK stack is a log management platform helps DevOps engineers, developers, and QA access and aggregate all system and application logs to make better decisions.
At Leapfrog, we use the Kibana dashboard to visualize all the logs generated from our servers and applications. The Lucene Query Syntax makes it easy to search for logs and to track the status of services on different servers.
The wealth of freely available software has enabled us to build amazing applications and solutions. We want to end this blog with a note from “The Pragmatic Programmer,” a legendary book about software development craft and learning goals.
Different languages and tools solve the same problems in different ways. By learning several different approaches, you can help broaden your thinking and avoid getting stuck in a rut.
Do you already use one of these tools, languages, and frameworks in your project? Let us know what open source project you liked the most in 2019.