Neurodivergent Loss Landscapes

Is applied ML work a safe activity for those with executive control issues? This is something I am coming to wonder about. Back when I first started competing in kaggle competitions, I used to find myself strangely drawn to watching the loss values for each batch tick in on the progress bar (using keras or mxnet at the time). Every time the value ticked down slightly I got a rush of dopamine, while when it ticked up my frustration and anxiety would grow. Had my latest efforts to make the winning change come to nothing? Did I know what I was doing at all? So much rested on this number ticking down. Even though the minute changes from batch to batch were so small and prone to noise that tracking in this manor was not helpful, I could not pull myself away to doing something more productive and just checking back later when there would be a change of meaningful significance.

Ten years later and I train LLMs as part of my day job. I still find myself glued to the loss values ticking in (in wandb now). The more important, urgent, abstract or open ended the particular task I am actually supposed to be focusing on is, like preparing for an important presentation, compiling a set of milestones or roadmap the more I find myself unable to avoid flicking back to the wandb tab to check on the latest results. I am starting to realize that these tendencies that been with me as long as I can remember are not experienced by everyone. There appear to be people who can actually set about a particular task and sit down and work through it to complete it in a reasonable amount of time. This dopamine chasing behavior is to be consistent with some forms of neurodivergence like ADHD. While I have not been officially diagnosed with ADHD, I scored pretty highly on the ASRS ADHD screener. It is a relief to know this is quite common and that it's possible to manage these tendencies and take back control.

Another trait of a person with ADHD is the inability to complete a project after all the challenging parts have been worked out. This one also rings very true for me, and it's one of the reasons I'm very excited about generative AI. If these less interesting parts of the project can be delegated to an AI agent, does this suddenly mean I will be able to complete things and unlock whole new levels of productivity. This to me is one of the most exciting things that generative AI can unlock.