Random thoughts throughout today

Nearly 3 weeks into 2019, & it already feels like a whole year has gone by.

What do you do when you feel like there's no more energy to get through the day? At mid-day?

How often should I take a break from work? Or from whatever it is I'm doing?

How much is enough?  How do you know?

Why is motivation so often intrinsic?


Quote of the day: 
In everyone's life, at some time, our inner fire goes out. It is then burst into flame by an encounter with another human being. We should all be thankful for those people who rekindle the inner spirit. -Albert Schweitzer, philosopher, physician, musician, Nobel laureate (14 Jan 1875-1965)

the MailChimp founders, a profile

A profile of the MailChimp founders:
https://www.forbes.com/sites/alexkonrad/2018/10/08/the-new-atlanta-billionaires-behind-an-unlikely-tech-unicorn


Keyhole surgery

A goldmine

https://www.r-bloggers.com/in-depth-introduction-to-machine-learning-in-15-hours-of-expert-videos/

https://raw.githubusercontent.com/asadoughi/stat-learning/master/ch2/answers

http://www-bcf.usc.edu/~gareth/ISL/

Skills for the next 5 years

Vala Afshar's outlines the transformation going on right this moment in time

Transitioning towards a data science career

These two articles made for good reading/ reference

Rafael Knuth blogged about his transition into a data science career. He inspired someone else to follow his footsteps, & write about his experience.

Comparing read_csv with spark_read_csv

Reading in a csv file into R using dplyr's `read_csv()` function is so simple. The syntax & parameters of dplyr are fairly easy to remember, once you've done it a few times.

read_csv(file, 
    col_names = TRUE, 
    col_types = NULL,
    locale = default_locale(),
    na = c("", "NA"), 
    quoted_na = TRUE,
    quote = "\"", 
    comment = "", 
    trim_ws = TRUE, 
    skip = 0, n_max = Inf,
    guess_max = min(1000, n_max), 
    progress = show_progress()
)



I've only just started working with big data sets, & was began wondering if what I know about the dplyr syntax can be carried over to sparklyr's spark_read_csv() function.

While not exactly the same, but if you know one, you can quite easily pick the other. There's an additional parameter `sc`, aka spark connection, that's required.

spark_read_csv(
    sc, 
    name,
    path, 
    header = TRUE, # FALSE forces a "V_" prefix
    columns = NULL,
    infer_schema = TRUE, # to infer column data type
    delimiter = ",", 
    quote = "\"", 
    escape = "\\",
    charset = "UTF-8", 
    null_value = NULL,
    options = list(),
    repartition = 0, # number of partitions to distribute the generated table.
    memory = TRUE, 
    overwrite = TRUE, ...
)

Inspirations for humorous speech contests

and John's Anatomy of the speech here


More movie based speeches - Johnny Cash - Walk the line 


and this one on any speech in general - David Henderson

Reading list - week ending 30 Sep 2018

https://stratechery.com/2018/instagrams-ceo/

https://fs.blog/mental-models/

https://medium.com/@timberners_lee/one-small-step-for-the-web-87f92217d085 & related https://solid.inrupt.com/

https://www.nytimes.com/2018/09/28/science/neil-armstrong-auction.html


https://www.bloomberg.com/news/articles/2018-06-13/amazon-s-clever-machines-are-moving-from-the-warehouse-to-headquarters

Asking Life's Questions

This ad from UBS was thought-provoking. A seamless flow of questions that nearly everyone can relate to.

Reading list - week ending 23 Sep 2018

https://www.nytimes.com/guides/smarterliving/be-better-at-parties

https://www.vanityfair.com/hollywood/2018/09/frasier-25th-anniversary-oral-history

https://www.nytimes.com/2018/09/05/well/family/why-you-should-stop-yelling-at-your-kids.html

https://www.theatlantic.com/family/archive/2018/09/spending-personal-finance-pain-of-paying/569575/

https://www.lifehacker.com.au/2018/09/dont-get-weighed-down-by-idea-debt/

https://www.theatlantic.com/magazine/archive/2017/10/kurt-vonnegut-the-drone-king/537870/

https://www.business-science.io/business/2018/09/18/data-science-team.html

https://www.forbes.com/sites/randalllane/2018/08/30/bezos-unbound-exclusive-interview-with-the-amazon-founder-on-what-he-plans-to-conquer-next/

https://www.conservethesound.de/en/

https://medium.com/@bryancostanich/the-future-is-tiny-44dea02e4517

https://www.nytimes.com/2018/09/23/business/media/the-markup-craig-newmark.html

Ray Dalio at Google

speaking about his book Principles. Fascinating talk.

Why bother writing here?

That is a question I ask myself every day. The answer, quite often, is "I don't know. Maybe I won't". And so I don't.

A few weeks ago, a colleague spoke at a meeting about feeling like an impostor. I think that is a feeling that I have lived with all my life. I know I don't know much, & I'm afraid all the time that I will be found out. So I say, or do, very little. And I suspect - no, I KNOW - that this fear has cost me an enormous deal.

Instead, I've been writing in my journal most days for the last 9 or so years. The journals I write in are a cheap ruled notebook I picked up in bulk for the kids to write. The words are banal, & describe very often my rather routine boring day. Other times, I'm caught in the emotion of the day, or the previous day, & it shows in my writing. The vocabulary changes. The handwriting changes. I just have to see the slant on the page to know how I was feeling that day :)

Over the last three years or so, I've gone back to my school days, & what I loved doing most - calligraphy. That focus has found itself showing up in my everyday handwriting. 

What's the point of this post?
Really, there was absolutely none. It was just a feeble attempt to get the words flowing again.

The hand of friendship

Yes Toastmasters Club celebrated it's 10th Anniversary with a simple, out-of-cycle meeting on Friday the 14th September.

No big deal, right? After all, Toastmasters is approaching its centenary.  There are hundreds of clubs that have been around for decades, so what's all the fuss about?

For me, it is. And I'm writing this down for my own memory's sake.

I'd moved with my family to Australia in 2009.  I'd struggled to find work, make friends & all the other challenges that a new migrant finds themselves in in the first year or so, sometimes longer. In my case, a chance meeting, about 4 months into this life-upending move, with someone who worked at Optus, turned into a 3 week contract, & then a  6.5 year stint there for me. A life-changing event but this story is about something else.

I'd been at Optus for a few months, when I realised that there was a Toastmasters club in there. I'd married into a family of Toastmasters, & had it inflicted on me previously :) so I knew what it entailed. So I decided to go along to a meeting. It was at work after all, lunch time, & I knew how welcoming these people are, so what I did I have to lose?

I walked down to the room that was listed on the campus info email. I don't think I've been that nervous even at my own wedding. The room was at the end of a long corridor, & there seemed to be a handful of people in there. Yeah, I can do this, I thought to myself. And then, I have no idea why, I had butterflies. What if they don't understand me? English is my fourth/fifth language, & these people are all going to judge me. Nope, not worth shattering my already frail self-esteem, & I literally ran back to my desk.

I did that for at least four months after that - every Wednesday at lunch time, I'd walk down that hall, pretend I was looking for a different meeting room, peer inside & then walk away. It was on my 17th attempt that as I was peering into the room, a man simply put his hand out, introduced himself, & said, "Welcome to Yes Toastmasters. Have you been here before?"  I have no idea if he'd seen me on my previous aborted missions, or if it was an innocent banal question.  I mumbled something, & sat down in that room that day.

There was no timer that day, so I offered to do the role. It is, for those who've never done it before, a fairly difficult role, especially when you don't quite know who's speaking, or the club's customs, but I remembered getting praised for my effort, even a little comment about how I seemed to be a natural, even though it was my "first time" :) Little did they know I'd done this many times before!

I remembered the feeling at the end of that meeting.  The group, made up of Aussies & migrants & second generation Aussies & for some for whom English was like me a second or third or fourth language were friendlier than I'd expected. In fact, a couple spoke to me for a while after the meeting, most likely missing their lunch.

I joined that first meeting simply to make a few friends, & grow my confidence in public speaking. That meeting changed my life.  It has done so for all my time there, & then some. I've served every year on the committee in some official or unofficial role. I've given tens of speeches, struggled through hundreds of table topics & given as many evaluations. And I remember none of them. What I remember though is the hundreds of people I've been privileged to listen to, meet, & in many cases, to call my friends.

And most importantly, I've never forgotten the feeling of having Suben reach out & offer a heartfelt welcome. It's something I've tried to remember at every Toastmasters meeting when we've had guests. It's become a part of me at every work or social event that I've found myself at, when I notice someone on their own.

I'd never told anyone about that first meeting. Coincidentally, it was in the same room that we were celebrating the 10th anniversary meeting. I stood up as the last impromptu speaker, & memories came flooding back. They gushed forth, leaving me choking. And I could feel everyone of the 30 or so guests in the room egging me on. Suben was in the audience, & I may have seen a pair of moist eyes.

Happy 10 Years Yes Toastmasters. And with the legacy you're built on, I have no doubt, you'll last a ten-fold more.

Writing to describe problems

A large part of problem-solving is the ability to see a problem in its true form. If you're able to describe a problem in a sufficiently clear and precise fashion, most of the solution will become evident.

Writing helps. Write about your problems. It's more precise than thinking

— François Chollet (@fchollet) April 22, 2018

On Keeping Promises

Are all promises worth the same?
How do you know?
If they aren't, how do you discern which ones are worth making, & which ones should simply be a "Sorry, no, I don't have the time to make it happen".

******************

Day 1 of my 2 week break has been fabulous.  Made a promise to myself that I'd not be working, & I'd be fully devoted to the family. Took the kids to the local library for term holiday activities that are usually set up. We had breakfast & lunch together, which was so much fun.  We bought a new phone for mum, & did all the chores we'd planned for the day. We played a game of badminton. It was unstructured, & it was fun, & it was liberating.

I did, of course, break my promise - kept browsing the phone, pining to keep abreast of the things I've set in motion before I decided to get off the wagon for a couple of weeks.  The phone turns off tonight.  Will I keep that promise?

Looking back

It's been just over two years since I started my role at Australia's largest infrastructure project. My title suggests I'm a finance analyst, supporting the revenue earning arm of the Government Business Enterprise behemoth that is the company. I'm about to wrap this two year stint to move on to another role shortly.

I've been asked a few times: what has it been like?

The early days:
The team I joined two years ago had three other analysts, so it was a really small group. Our manager has a reputation for being one of the better managers around, and it was quite evident right from the first week. 

It so happened that I joined right in the middle of the annual long-term forecasting cycle, so I was right in the thick of things. Despite my previous experience in a similar industry, the  tasks I was assigned were new, different & frankly sometimes quite obtuse. I also had a regular role that I was to shoulder, as well as take over the weekly & monthly reporting. To top it all off, one of the team was going on leave for three weeks so it was a definite baptism by fire.

I did have a few things going for me:

• I knew many of the people I needed to interact with (staying in the same industry helps enormously!) or could fairly easily build a relationship with through introductions from mutual connections
• Data wrangling & analytics were two of my secret weapons & I wasn't shy about using the right tool for the job. I also knew my way around the IT department, thanks to a couple of my good friends who'd preceded me into the organisation. 
• I do have a commercial background after all, so I knew precisely what the numbers I was reporting would mean to the recipients

So while the first few weeks felt overwhelming (I also got the distinct impression that I was being pushed to the limit), I had landed on my feet within a couple of weeks, & was learning the ropes quickly.  My manager was regular with both feedback & praise, building my confidence & helping me get very clear with what I needed to deliver. 

It's been my personal mission to learn something new every week.  While still early days in my new job, I still kept up with my learning. I had also just finished reading Nancy Duarte's blog post on slidedocs, so was messing around with the corporate slide templates the organisation had recently issued to see if I could use them more effectively in my work. I had just started reading about the Google Places API. I wasn't quite sure if I would ever use it but it seemed interesting to me. 

By the end of month one, I'd put up a template (with some VBA) that made preparing & presenting the long-term budget a far-more effective & efficient task. The solution was simple: make the presentation a book-like form, laid out in landscape, with the even-numbered page showing the graphic & the odd-numbered page showing the numerics. While there was some initial resistance (finance processes, anyone?), the exec team loved it when shown a draft, & it was a resoundingly successful production effort. 

Less than 2 months into my primary job supporting a team with commercial reporting & analysis, I was told of the peculiar challenge that this team faced: they had no idea who their end customers were. It was like a light-bulb went off when I heard that.. why couldn't they use Google Places API? I spent the next few train trips to & from work using R (another tool I was teaching myself to use) and the Google Places API for R building a prototype that might possibly be a solution.  I shared my prototype with my manager, who immediately sent it on to the head of that team, but also reminded me that it was not my job to solve that specific problem.  Let's just say that prototype, & a few follow up actions on my part, earned me a trip to Google offices in Sydney. 

note to self:  Solve problems, even if they aren't in your specific job description. The value gained personally in the learning far far outweighs the politics of the situation. 

By the time I had completed my official probation period, I had already made my presence & value felt around the organisation. I'd saved a bunch of people a few hours of work, & not just in reporting, showcased some analytical capabilities, & build a bunch of reporting frameworks that made reporting very boring :) 

What does a good manager do? Find a challenging project, & throw it the bored employee's way. Alongside my day job, I was thrown into a pricing project that was struggling for modelling skills. Three months or so of intense optimization modelling later, the proposed pricing structure went into the market and dramatically changed the industry's behavior, but without endangering the revenue prospects of the company.  I value the learning on this project hugely - the mental models of thinking about problems, the approaches to solving them, & the wide variety of factors that were considered in the final solution, all within a deadline that seemed insane at the time. Most importantly, the relationships that were built in that project have grown into friendships. 

 

I've since been involved with some mind-mending business problems. I've been thrown into projects that seem irredeemable in the time available to solve them. I've learnt to see the problems of a behemoth operating at scale in many different angles - regulatory, commercial, operations, strategy, financial, and human. I've seen many of those angles considered and ignored, ideated & forgotten.  In any case, I think these last two years have shown me how much I can accomplish in a very large organisation, how much I can collaborate with people far smarter than I will ever be. I've also learnt about (& felt the impact of) the politics that accompanies a large organisation, of rewards & recognition (or the lack thereof), of personal & team motivation, of technological opportunities & technology's self-inflicted wounds, and a whole heap of things that won't ever make it to a blog post. 

The discipline of finishing

It must be a common trait - we begin doing something, & before long, it is left by the wayside,


*************

Ironically, I started writing this first sentence a few months ago, & didn't do any more than those words!

Tools of the trade...

For someone who works in finance & commercial, Microsoft's Excel & PPT are the glue that hold most organisations together. Forget the multi-million dollar BI implementation approved, ironically, by the CFO, when some analysis is required, its 'open an Excel spreadsheet' time every single day.

This week I found myself talking to a grad in my team about the tools worth getting familiar with, & I shared with her how my thinking has evolved over the last two decades:

Stage 0:
What's this Excel thing? And who on earth needs a million rows & 256 columns?

Stage  1:
Wow, I can get some pretty decently presented tax computation worksheets. The IF statement, especially those 6 nested conditions cover nearly all my likely needs.

Stage 1.1:
Why didn't someone tell me I could use VLOOKUP  instead of those nested conditional statements?

Stage 1.2 to 2:
There's so many more formulas? And you can combine them?

Stage 3-7:
Whoa!! Alt  + Ctrl + Shift + number keys + arrow keys + Page Up/Down keys can move me around the worksheet/ workbook faster than moving my hand to the mouse & then clicking!
Ok, time to learn to touch type because I can do things at x times the speed of mouse clicks.
Formulas, referential indexing, Data Analysis add-in.. holy f**8!!!! Pivot Tables, External Data, connecting to databases, MSSQL, formula auditing, charting......  there's more stuff here that will make my reporting life easier than ever..

Stage 8:
What's this 'macro' thing? It's totally pointless, does all sort of nonsense when I record something & then push play..

Stage 9:
I can take someone else's macro & copy it & then change it to suit my needs? I can integrate it with OLE & ODBC & SAP & Hyperion & other VBA reference libraries to automate some of my daily tasks? What am I going to do with all this spare time?

Fast forward several versions of Office release later, & all the power I need for my day job at my finger tip memory (PowerPivot, PowerQuery, PowerBI),  - there's no need to learn anything else... & well, Tableau came along that made visual analytics easy as point & click (grrrr, no finger memory possible, but I think that is deliberate to make you think while you click around dimensions & measures)

So if you're starting out in your career, it's probably best to get really good at three things with your tools:

1. Touch Typing (there's probably reams written on why this is important). Keyboard shortcuts in Excel will save your a$$ at midnight when all you want to do is shut the damn thing off instead of reaching for the mouse once again. My experience is that you will not even be working at midnight once finger memory takes over.

2. Navigation shortcuts using keyboard.  Get as familiar as possible with:

• Windows application keyboard shortcuts (that Win key + numbers can fire up the application you need)
• Excel keyboard shortcuts (and don't bother memorising these - watch the screen light up when you hit the Alt key). 
• Any other application - before you start playing with the application, read through the help file for keyboard shortcuts  (yes, even Tableau has a few!)

3. Get at least a basic understanding of how code works. You don't need to be a coder or programmer (you hate it so much & it also explains why you'd rather be an accountant :)) - but knowing how the thing you work on works (it is software after all!) will give you a few ideas when you're struggling to do something that seems nearly impossible (which will happen soon enough, anyway)

As the creator of D3.JS Mike Bostock says: "Code is the most general tool we have"

Progress... slow, but progress

The last few days, I've been spending a lot of time on teaching myself to think through problems in a different area.
The course on Deep Learning is pushing the boundaries of my comfort zones.
Every day, however small the progress may be, it feels like a breakthrough.
Waking up way before dawn again today, I spent just under two hours tackling the implementation of backward propagation in python code, vectorised no less.
It was very very hard going for me, yet, after I submitted the assignment & saw the score, there was a tiny sense of accomplishment.

That feeling, amid all the weariness, is very valuable to me at this point.
That, & the music I surround myself with.

Of course it is only month-1/ course-1 of a 5 part course, so things will only get harder from now on.

Public transport conversations, & a minor personal breakthrough

A lady sat down next to me on the train this morning. She was already on the phone when she got on the train. I knew because she had a distinct voice that carried. She was 'coaching' someone, from what I could overhear one side of the conversation.

Two things stood out: In the course of her conversation, she advised her 'mentee' that someone had to just go out & talk to people, & just do some work. There was no question about why that was needed, just that something needed to be done.

The other was her story of a conversation she had with someone a few years ago, on dealing with stress. The person in question worked at a hospital, & would walk around the place, to get a perspective that whatever was causing them stress wasn't causing people to die.

*******************

I've been putting off doing an assignment on the Deep Learning course because I simply couldn't get my head around what seemed a basic concept. The lack of motivation in the last couple of days was partly that too, perhaps?

Anyway, after dinner, & reading to the 6yo, I approached the problem using paper & pen. It took an hour or so to draw out the concept, & about 10 minutes to get through what seemed to be the hardest part of the assignment.

Which reminds me, I used the same technique at work today to design a report that I'd been putting off.  Broadly, the #Tableau report needed to keep track of where each customer lay on a price curve, with the price table axis at irregular intervals. Writing out the steps in full make it amply clear where the problem was & how to solve it.