Struggling to Find the Right Fit for a Data Role? Try Easing Up on the Cloud Jargon.

Shôn Ellerton, Feb 24, 2022
Finding the right candidate for a data role is much easier by asking about key data skills rather than obsessing about the Cloud and other esoteric software.

Why is it so often difficult to get the right candidate for a data role?

The answer: Data architects, cloud computing, and role briefs with unrealistic expectations requiring candidates that they know every esoteric third-party software known to the IT world.

Data architects have their place as does Cloud computing so before anyone has a chance to brand me as being stuck to my ways and being ‘old school’, let’s talk about them and why they’re often a pain in the backside when it comes down to IT interviews. Let me say for the record that I am a data architect guilty as charged but not always in a role as one.

To start with, what is it with this obsession with cloud computing? Ailing companies suffering with IT issues rushing to their HR recruitment teams begging for cloud experts with a fancy title like systems cloud super-duper-look-at-me-I’m-more-important-than-a-coder configuration senior analyst or any number of a myriad of other lofty titles to solve all their IT problems. Role briefs to prospective candidates reading off a lengthy tome of third-party cloud products, most of them being software-as-a-service (SAS), some with names like Snowflake (not to be confused with snowflake schema-based data warehouses or of someone described as being one) or cool space-like names like Amazon’s Redshift. There are so many of them and, no doubt, few would have experience in knowing how to use a small fraction of them, let alone several. There are many data people out there who may not know any of them. Yet they may possess a fine set of skills in handling data and designing databases.

Let me tell you what happened to me on two occasions. It was the type of woolly rhetorical interview in which I’ve been asked a series of questions on what I would do, should I be accepted into the role, to remedy their problems. To which I offered my advice only to later find out that they said that they are not filling that role anymore or that they are rejecting me on grounds that they have already found a candidate more in line with their salary expectations and, wait for this, for me to find out later that the company did take my advice. I’m not making this up!

Let’s start with HR, or People Management, or Partner Alliance Employee Integration Services or whatever they’re called these days. We don’t expect them to understand the details of what cloud computing is or what data management is all about, and that’s fair enough. But at least one person in the interview most certainly should have some technical knowledge in what they are advertising in the brief and, moreover, they should be asking the right questions.

For example, there was a job posted for integrating data and managing several instances of database servers and all the basic stuff needed to ensure that all the data flows are functioning properly, and internal clients are given the resources to produce the reports they need and so on. Towards the bottom of the brief, it mentioned that cloud computing would be a desirable skill set. The brief also mentioned that having any experience in Hyperion, Amazon Redshift, SAP, Snowflake, Databricks, COGNOS, Power BI (almost always), Tableau, and many more exotic sounding technologies which I can’t remember off the top of my head, would also be desirable for the role. Then you have all the other data technologies which are, to be frank, hardly used in most small to medium-sized businesses, examples including traditional cube design for data warehouses, graph databases like Neo4J and a multitude of ‘big data’ NoSQL variants like Hadoop and Mongo DB.

I spoke to my agent and said that, in all honesty, they probably need someone with a blanket set of skills across these technologies. I explained that I know the tenets and precepts of data management, how to put together databases and data warehouses, how to move data from one place to another and said that I had an advanced knowledge of SQL and reasonable skills at understanding how to code in other languages like Python, PHP, and C#. The agent outlined my response in the most predictable way agents know of by explaining that what the client is looking for is someone who is flexible and adaptable, has the ability to learn and able to understand the principles of data management. Kind of the wishy-washy stuff that HR people like to drone on about. So, I applied and got accepted for an interview.

I turned up at the interview, and after all the pleasantries were over with, I sat in an uncomfortably small room with an oversized table along with three other people. One from HR, one from business development, one being the IT director and the other, surprise surprise, a data architect. Being a data architect myself, this flagged two concerns.

First, data architects, much like proper architects, are highly territorial and genuinely suspicious of other architects. Like a praying mantis, they will fight to the death. Despite my attempt to alter my CV to make my skills fit with the desired role by focussing on ETL, data management, SQL and some cloud computing, the data architect in the room stumbled upon my LinkedIn profile which said, at time of writing, that I was a data architect. (I really need to amend my LinkedIn account, shouldn’t I?)

Second concern is that data architects tend to start jettisoning off at a tangent towards the Oort Cloud (I jokingly use this term for someone in a daydreaming kind of state) in probing any candidate’s knowledge in esoteric cloud-based systems while spruiking that the business is doing a ‘complete re-think’ on its IT infrastructure. Basically, this is doublespeak for when a business is in a complete mess with respect to their IT and have no idea what to do positing the general idea that anyone accepting a position there will probably not last very long. And while this is happening, the IT director, a very important position no doubt, will be nodding his (or her) head in solemn approval much like what the background guys do in a politician’s speech.

Hiding behind a friendly smile and of a professional deportment, the data architect starts off with the most predictable question on the block.

“What’s your experience with cloud computing?”

Never mind the questions about my other more pertinent data skills, after all, once two data architects are held in captivity within the confines of a small room, one’s got to go, and that, of course, will be me. At this point, from prior experience with such interviews, this usually signals the death knell in terms of securing the role.

I will make an interesting observation that in those interviews in which a programmer—or coder, if you will—is present, there is a much higher chance of getting the job as they tend to ask the more basic down-to-earth questions the other people in the room can understand. The coder often highlights the immediate issues which need addressing rather than the long or medium-term glorification plans the data architect has in mind. IT managers and directors want, in general, business-as-usual (BAU) activities to go on smoothly whilst on their watch, which, of course, is in their immediate and best interest. Conversely, those candidates who talk with brimming self-confidence and impressive jargon—albeit, with little practical substance—playing word bingo trying to get in as many terms as possible in the conversation as listed out in the brief may certainly captivate the others, but possibly not the more down-to-earth programmer if I’m allowed to stereotype somewhat. Coders are not bullshit artists in general.

Going back to the question of someone’s experience of cloud computing, that is all well and good but when the more important questions are not being asked of experience with databases and data warehouses, ETL, SQL programming, and all that core stuff needed for data management, it strikes me as being absurd. I’ve worked on various cloud-based platforms including Azure and AWS managing SQL Server databases and BLOB storage management at some place of employment or another and it goes without saying that without the core skills required for understanding data, all that cloud stuff is as useful as growing a third eyebrow or selling fridges to Eskimos.

If you were hiring a truck driver, would you not be asking about that person’s experience in driving a truck rather than where? Sure, driving a truck in England is bound to be different than driving a truck in outback Australia, and this same logic can be applied to working with data using on-premises systems and cloud-based systems. You still need to ‘drive’ the data, albeit there are some additional things to learn as well with Internet connectivity, business-rules security, usage tariffs and a lot of other stuff which the cloud differs from on-premises systems. On the other hand, one doesn’t have to do all the infrastructure, server management, backups, security, and patching that comes with on-premises systems.

The problem which many out there who work in the enterprise solutions or data architect space do not realise that many of these new and wonderful cloud technologies turn out to be more expensive than a traditional system because it often requires a more specific set of skills in that technology. Other factors include ill-thought-out subscription models and poor contractual arrangements, an example I recently came across when a company I worked for spent a significant sum of money to release in bulk format certain data held by the vendor not covered in the contract. The cost, which was massively inflated, was the extra resource required by the cloud provider to facilitate the movement of this data. Many of these shiny new tools are sold to business executives by exciting and inspiring demonstrations from well-versed software salespeople promising that they have every solution to the exacting needs of your business. Naturally, those data architects and solutions experts want to show off these wares and obtain funding in their businesses to do so. Now, to be clear, I’m not suggesting that using cloud services isn’t the right way to go. I’m just saying that there is a popular school of thought parading through IT recruitment circles that the only way to go is the Cloud. There are cases for on-premises, cloud-based systems and hybrid systems depending on the business model. It just depends.

However, there was a silver lining to the cloud (yes, I know, a cheap pun) when word came to me through an agent that one of their clients found it frustrating to find a candidate because they were, indeed, asking the wrong questions, although, I wasn’t looking for work at the time. Instead of looking for someone who had experience with every item on a long list of specific third-party exotic cloud-based vendor products, they were simply looking for someone who had a good knowledge of the basic skills required to manage data. Out of curiosity, I asked my agent if they eventually found somebody and sure enough, they did. I made a further enquiry from the agent about any feedback of how things went with that person. I was told that everything was going smoothly for him. But what was most noteworthy was that he came in with relatively little knowledge of cloud-based systems. There are third-party systems which genuinely require a great degree of experience, but most can be learned quickly provided the candidate has a grasp of data skills. The company took the sensible line of approach in questioning his knowledge on how to manage data rather than his experience with a list of products which the candidate, once settled, can quite easily learn on the job.