Mental Illness and Obsolete Skills
October 2019 was the beginning of a very difficult and dark period for me. In truth, the decline began like a slow-motion train wreck during Christmas of 2018, but the main crisis point and collapse was triggered in October 2019.
In October 2019, I discovered that I had a serious mental health condition that had been undiagnosed for more than 30 years and that my almost 15-years worth of SAS skills were no longer in demand and likely becoming useless. The following story is my journey to mental health well-being and to modern data science methods – two threads of the same story:
First, the mental health journey
Although being told I had a serious mental health condition that had likely been undiagnosed since my early teens was very shocking and upsetting, the knowledge of my condition allowed me to take appropriate action to manage my condition – at least that is the theory … but … as the saying goes … you can lead a horse to water …
Even though I was made aware of this condition, the only ‘appropriate’ action that I wanted to take was inaction, social withdrawal and isolation. I resisted all attempts at help even from my closest family members and friends. I was in a very dark place and seemed quite ‘satisfied’ / willing / determined to remain there.
After going nowhere fast by tucking myself away into a dark corner of the world and hoping to disappear – shocking right? With encouragement from family, friends and that great occupational therapist, I began counselling, occupational therapy, meditation and mindfulness. I began to feel better. The dark clouds of obscurity began to lift.
Passing from the darkness and abyss of obscurity into the light of knowledge can be painful, frightening and very frustrating. An open mind and a positive attitude along with a supporting team are critical.
“Knowledge together with will power and support leads to action and improvement”
Now The SAS To Spark Journey
My SAS-To-Spark transition mirrored my mental health journey. It was initially shock and dismay followed by resentfulness, frustration and anger then helplessness and hopelessness and finally resurrection and progress. Do not succumb to fear – especially fear of the unknown!
SAS Fading Away to Oblivion?
Since 2006, SAS has been my main bread-and-butter BI tool. I invested much time and effort learning and enhancing my SAS skills. I used my SAS knowledge and skills to shift careers from automotive quality engineering to the Management Information / Business Analytics / Business Intelligence / Data Science field that I have been in since 2006.
I achieved Base and Advanced SAS Programmer Certification in 2009. I used my SAS knowledge to work overseas in the UK in the British banking sector. After almost six wonderful years working in England, I returned home to work with a major Canadian telecommunications company. Finally, I went to work on a medical informatics information delivery team, with a major healthcare insurance provider in the southeastern US. I had built my entire BI career on SAS! I thought I was safe with my SAS skills as they were.
But the winds of change always blow …
In late 2019, my employer at that time had started to explore using open-source tools like Python and R for predictive modelling. At around the same time, a dispute occurred between my then employer and the SAS sales account management team over licensing fees. A modernization effort began to transition away from SAS to open source technologies.
Oh no!!!
The existential questions begin racing through my head:
- Was SAS no longer going to be used as the BI and data analysis?
- Would open source platforms like R, Python and Apache Spark replace SAS?
- Was SAS being phased out?
- Were my skills that I had spent more than 15 years cultivating going to be made obsolete?!
- What am I going to do?
Initially, I tried to keep up with the changes by taking all the Python, R and Spark courses offered by my employer and also available on Coursera and Udemy. But, I was completely overwhelmed and shell-shocked. Nothing stuck! I wanted to disappear. I slipped into that dark place and I felt like I was staring into the abyss. I withdrew from most social activities. I did not even like doing SAS and SQL programming and development anymore. Then of course, along came the COVID19 pandemic and things got worse! I thought it was the end …
I was wrong.
“The only thing we have to fear is fear itself” – FDR
It was not the end – far from it. Towards the end of 2020 and into 2021, I began to rebuild my confidence and my desire to learn my refocusing on online Spanish lessons. I had been studying Spanish on and off since about 2015 but I signed up with Babbel and went all out and doubled-down on improving my Spanish language abilities. I began contacting and corresponding regularly with my cousins in Argentina – for the total immersive language experience.
Then came my resurrection trip – a one-month adventure in Colombia! Que bueno! After this trip to Colombia, I was well on the road to rebuilding my confidence and opening up my mind to learning new things in general – whereas previously my mind felt like mush!
So, I began to study Python again … and it was not so scary anymore! It began to make sense to me – slowly but surely. As I worked through the Python texts that I had purchased in 2019, I began to appreciate its Python’s many advantages – its elegant simplicity, its scalability, its power. In addition to starting up again my studies in Python, I started to take Coursera courses on Cloud Computing – specifically the MS Azure certification track preparation courses.
Improvement was happening slowly but steadily but the best was yet to come …
Where I saw hopelessness, Ian saw golden opportunity….
Then I received a divine blessing like manna from heaven. Through a recruiter friend that I have known for 10+ years, I was introduced to Ian Ghent and his amazing data science start-up WiseWithData.
Ian and I had similar backgrounds. We both started in the BI/Data Science field using SAS and the main analytics tool. We had used SAS in various organizations like banking, telecommunications etc. – Ian had even worked with SAS Canada.
Right around 2015, Ian began hearing rumblings of SAS losing market share to open source tools like Python and Apache Spark (PySpark). He heard about SAS upper management acknowledging that these open source tools were an existential threat to SAS. Then came increased turn-over, layoffs and a planned IPO – things that SAS – particularly its founder Jim Goodnight – committed to NEVER doing. WHISKEY TANGO FOXTROT
Ian reached the point in 2015 that I reached in 2019. Whereas I saw hopelessness, helplessness and the abyss, Ian saw a tremendous golden opportunity to help others make the transition…
Ian began to research open source tools, particularly Apache Spark and in his research, he found Spark API’s that mirrored functionality in SAS, and enabled the parsing and translation of code. From this was born WiseWithData and the SPROCKET Solution – a solution to automatically convert SAS Legacy Code Processes into modern PySpark. A powerful technology to help any organization modernize from SAS to advanced cutting edge open source data science technology.
As I spoke with Ian, I was more and more impressed with his vision and with the opportunities at WiseWithData. In me, with my extensive SAS experience, he saw the type of knowledge and experience to help guide development efforts to enhance the SPROCKET solution to make it more effective, efficient and user-friendly.
For me, the opportunity was one of a lifetime. I had been given a role, in which I would use my SAS knowledge while learning and strengthening my Apache Spark and Python skills, all to help motivate and transition customers from legacy SAS to modern Data Science tools.
Brilliant! Best of Both Worlds!
Since learning of WWD’s mission and product, I’ve become convinced that PySpark will be lingua franca for all analytics. It has been a tremendous and amazing journey and I am so much looking forward to where WWD will go.
To all my fellow SAS power users
Do not be afraid of open source technologies like Python and Apache Spark. Embrace them! Your grounding and background in SAS will serve you well and learning the new technologies is exciting and fun but more than anything – goodness gracious is PySpark ever FAST! It is also scalable, flexible and works in almost any environment. It is very easy to learn and to use and the open source community support for it is large and growing exponentially. New features and functionality are being added on almost every day!
The transition from SAS to PySpark can be challenging but it can also be very fun and gratifying. To help you transition with this transition from SAS to modern data science methods, there is no better partner than WiseWithData.
Learn more about our SPROCKET SAS Modernization Solution – [email protected]