What the "Age of the IC" Means for Data Professionals

Hello Reader! I just got back from my trip to Australia. Besides enjoying the delicious coffee there, we still continued to use math and AI while down under.

Last week, we went to Australia for my college undergraduate son to present two scientific papers at the 8th International Meeting on Origami in Science, Mathematics and Education (OSME8) in Melbourne.

We then extended our family trip to Sydney. At the behind-the-scene tour of the Sydney Opera House, we learned that the iconic building was designed with perfectly spherical sections for easier construction.

While in Australia, we found that the AI in the Merlin bird-identification app from Cornell was unable to identify the local birds by sound, but still managed to identify them visually. My son and I assumed the app didn’t have enough audio dataset to train for Aussie birds.

Overall, I’m grateful for this opportunity to have visited Australia and get refreshed before resuming career development for people.

What the "Age of the IC" Means For All of Us

Upon returning from vacation, I saw that Dan Hockenmaier, Chief Strategy Officer at Faire, recently posted on LinkedIn:

He explains his reasons:

His post sparked a lively debate about A) whether this condition is cyclical or long-term, and B) the role of AI on IC promotions.

I do agree with him that organizations are placing higher expectations on workers to produce more with less. Here’s one possible future that I anticipate:

• ICs will evolve from Individual Contributors to Independent Contributors. ICs will get less managerial support and company support. They’ll effectively become a free agent, almost like an independent contractor, or entrepreneur, within the enterprise and in industry, as they use AI to create and sell new products.
• Managers will become enforcers of company policies and priorities. They’ll need to judge and prioritize people and projects as they enforce accountability within their part of the company. Managers will enjoy less creativity and autonomy as responsibilities get shifted to ICs within the broad teams. Managers will need to prove their own worth in new ways as they inherit broader teams.

ICs and managers alike will need to adapt to the new expectations of their roles.

Advice for IC Data Professionals

For ICs in the data field, such as analysts, data scientists, and AI/ML/data engineers, I offer some career insights gleaned from the data from DataNerd.

​Luke Barousse from Louisiana had built the DataNerd.tech website to analyze data on desired technical skills for data analyst job openings. He had collected over 2.7 million open job listings for data professionals from the internet. Initially he had scraped LinkedIn until he got banned. He later switched to using the SerpAPI service to download job listings that Google had aggregated from LinkedIn, Indeed, and other sources.

His UI is simple, with some data anomalies, and sometimes errors out. All those issues can be forgiven though, as he was resourceful in solving a specific need as he shipped his free solution out to the world. For transparency, his code is in GitHub and his data is on Kaggle for anyone to review.

For an example of the type of data insights his tool provides, see this screen capture. His data tool shows that the top 3 skills listed in new US job listings for Senior Data Scientist in 2024 (January to July) are: 1) Python, 2) SQL, and 3) R. These 3 skills are consistent with my experience with data scientist roles and with my own previous data research while at LinkedIn.

Overall, for the 6 non-senior job titles in his data, the top skills intuitively make sense. I summarize his data into the below chart for you.

His DataNerd site also shows the trend of the technical skills desired by employers since 2023, as well as the median salaries advertised with the job listings. As the salary data has a lot of variability, use it for directional guidance only.

I analyzed the data reported on the DataNerd site. Based on this data, I will recommend the following career advice for you based on your current role.

• Business Analyst: You should already know SQL and Excel/PowerPoint/Word. To grow your career, learn Tableau/PowerBI and Python.
• Data Analyst: You should already know SQL, Excel, and Python. Grow your career by learning Tableau/PowerBI too.
• Data Scientist: You should already know Python, SQL, R, Tableau and possibly AWS. To grow into a Senior Data Scientist or for a higher salary, learn Spark, PyTorch, and TensorFlow. These additional skills are also needed by AI/ML engineers.
• ML Engineer: You should already know Python, PyTorch, TensorFlow, and AWS. Learn Spark and SQL for more versatility in building data pipelines.
• Data Engineer: You should already know Python, SQL, and AWS/Azure/Spark. To grow into a Senior Data Engineer or for a higher salary, learn Scala, Snowflake, Redshift, and/or Databricks. If you want to go to the cloud engineering route, learn HashiCorp Terraform for infrastructure-as-code and Ansible for cloud configuration management.

I also observed from the DataNerd data an emerging trend that employers more likely desire TensorFlow and PyTorch skills than Tableau skills from Senior Data Scientist hires since March 2024. This trend corroborates with what I’ve seen with employers wanting their data scientists to step up to build AI/ML systems, while business and data analysts step up to use Tableau.

The above recommendations based on the DataNerd data are for technical skills only. Certainly other skills like leadership and communication are always desirable too, but they’re not listed in the DataNerd site. (see Top 5 Valuable Skills to Future-Proof Your Career)

Advice for Managers of Data Professionals

As Dan Hockenmaier asserts that we’re entering the Age of IC, we see managers also forced to adapt. Managers of data professionals are challenged with:

• higher expectations with more limited resources
• more churn and reorgs and reprioritizations
• enforcing unpopular company policies of return to office, budget cuts, and more rigid employee performance management
• and all while trying to deliver ROI on AI initiatives.

BTW Goldman Sachs has recently pivoted to being openly skeptical about the ROI of AI investments by companies. (see Goldman Sachs Gen AI: too much spend, too little benefit?)

For the unique challenges faced by data leaders, I’ll share more advice in a separate email. In the meantime, please consider joining our next cohort of the Data Leaders Community of Practice.

Data Leaders Community of Practice Cohorts Starting September

If you are a new, or a long-time, people manager of data professionals (analysts, data scientists, AI/ML/data engineers), and would like to get practical advice and support to become a better data leader in your company and in industry, join one of our upcoming cohorts.

We designed the Data Leaders Community of Practice cohorts exactly for you.

Learn real best practices to solve your challenges confidentially with other data leaders throughout industry.

We are starting two more 6-month cohorts of our Data Leaders Community of Practice in September. Based on requests from people outside of the SF Bay Area, we will also launch an additional online-only cohort for remote US participants this time.

• Cohort #3 starting Thu. 9/5/2024 (online-only)
• Cohort #4 starting Fri. 9/20/2024 (SF Bay Area)

Visit the website to learn more and to sign up. Sign up now with the “JIMMY” referral code to get a 10% discount.

If you have any questions about this cohort-based coaching group, or want to find out about private individual career coaching options with me, please email me at [email protected] to schedule a call.

Jimmy Wong Coach, speaker, and entrepreneur enabling people to thrive in the age of AI. Data science leader with 12 years experience at the LinkedIn company and 27 years in the industry. Visit aijimmy.com​ ​​​​​​

​