It’s been almost three and a half years since I published “Database Testing Patterns in Go“. The post was about how at clypd we were using dependency injection to test functions that would access external data sources. It’s a testament to the effectiveness of the pattern that it lasted so long. However, as our code base has grown over these years, we eventually started to run into some growing pains associated with the way we were doing things. Read More
clypd’s workflow involves offline processing of files from our clients and partners. They drop compressed ZIP files to our SFTP servers which are then picked and processed by our workers. The processed data is crucial in powering clypd’s advanced audience targeting platform. Because these files are critical to our business, we have monitors that ensure correct parsing and transformation of the files.
What do you do when your parents pull you out of school to go to Disney World but you have a project due the next day?
However, there were a few issues:
At clypd, almost every feature we build is data-driven. It is therefore important for the data access layer of our Go platform to be powerful, yet easy to build upon, maintain, and debug. Today we dive into the design of the clypd data access layer, the frameworks we chose, and how we augmented them.
A huge benefit provided by Go is simplicity throughout the clypd stack. Dependencies, particularly third-party libraries, are often opaque in other programming languages due to varying code style, source code that is difficult to find, or abstraction layers that are challenging to see through. The source code for any library is just a click away in Go. Design idioms found in dependency libraries are usually similar to the ones we use, which ensures that the entire system is easy for our team to comprehend.
At the beginning of 2014, I was thinking about how we might create a meaningful review process. I am not a big fan of only yearly reviews. I don’t know about you, but I have no idea what I was doing a year ago and often, what was important at this time last year is not relevant now. So, how do we make a better process that helps drive career growth?
I already had a one-on-one process established where we have the “how’s it going?” discussion. I always start the conversation with “Are you having fun? Are you learning stuff? Are you building awesomeness?” The response I always get is “yes, yes, yes.” Then I ask, “What did you learn? What part was fun and what was not? What kept you from building awesomeness?” And that’s where the interesting conversations start. For me, learning, fun and building are intertwined.
We’re excited to announce that clypd’s VP of Engineering Jeff Walker is a Timmy Award finalist for Boston’s Best Technology Manager!
The Timmy Awards is hosted by Tech in Motion, a national events organization for tech enthusiasts created by Jobspring Partners & Workbridge Associates. The 1st Annual Timmy Awards recognizes the best places for technology professionals to work. Categories in The Timmys include: Boston’s Best Technology Manager, Boston’s Best Technology Work Culture, and Boston’s Best Tech Startup. To see all the finalists, be sure to check out Tech in Motion.
Almost a year ago, we blogged about our reasoning and methodology for choosing Go as our next generation platform here at clypd. A year is a long time, both in software technology and in the lifetime of a startup. Since paper is the traditional gift for a one year anniversary, it seems appropriate to write down our learnings so far.
When we posted “Getting to Go,” we included a list of the selection factors we used to evaluate potential platforms. The list includes
Thorough testing continues to be a key tenet of software development at clypd. Quality is paramount, but shipping code quickly requires us to be efficient in how we search for bugs. We’ve found innovative ways to unit test individual Go packages of our programmatic television ad platform as it has grown in its capabilities. The construction of a similarly efficient mechanism to test large portions of the whole system has taken our test automation to the next level.
At clypd, we place a lot of value on testing as a mechanism for ensuring code correctness and our approach to testing constantly evolves. We’ve spent the past few months learning and creating a new approach to mocking and testing functions that need to access data, whether from a database, a file, or over the network.
At clypd, we have been using PostgreSQL and it has been a great experience. We are impressed with its simplicity, good documentation, active community, abundance of functions and lot more. This post jots lessons learned and insights acquired during a recent journey in improving performance of one of our queries. Most resources and tools in this post point to other sites, however we think it would be helpful to list various ‘performance improvement’ resources in one page. Read More