Sign in

Shawn Hymel

@shawnhymel.bsky.social
1.8K followers 955 following 469 posts

Embedded Systems Educator & Course Developer | #IoT #EdgeAI | linktr.ee/shawnhymel

PostsRepliesMedia
Shawn Hymel @shawnhymel.bsky.social · 06/10/2026
Apparently, not knowing what “unc” means is very unc.
050
Shawn Hymel @shawnhymel.bsky.social · 05/10/2026
Had an amazing time at my 20 year reunion! My roommates/friends visited. We rented an Airbnb, played board games, and reminisced about various classes/activities. I love how old jokes immediately resurface when you meet up with old friends ❤️ @rosehulman.bsky.social
020
Shawn Hymel @shawnhymel.bsky.social · 22/09/2026
Anyone need some #math on their Tuesday? In my latest #ReinforcementLearning post, I cover Generalized Advantage Estimation (GAE): how λ-weighting blends TD and Monte Carlo advantage estimates, and how rollout buffer length affects it in practice. #AI #MachineLearning shawnhymel.com/3748/reinfor...
shawnhymel.com
Reinforcement Learning Part 16: Generalized Advantage Estimation - Shawn Hymel
While we looked at the surrogate objective in the previous post, we’re actually going to return to the discussion of advantage functions from part 14. In this
010
Shawn Hymel @shawnhymel.bsky.social · 20/09/2026
Haven’t painted in a while, so it was nice taking a break today to knock out the white dragon scale shield for my #DND character 🎨🖌️ #miniaturepainting
Goblin mini with a scale shield
0283
Shawn Hymel @shawnhymel.bsky.social · 17/09/2026
Had an amazing time out at Grand Junction and Palisade. Gorgeous area of #Colorado with amazing views, cute wineries, and, of course, 🍑s.
041
Shawn Hymel @shawnhymel.bsky.social · 03/09/2026
My final episode on #ReinforcementLearning for #robotics just aired! We add remote control commands in order to drive the balance bot around. Check it out! #AI #Arduino #engineering @digikey.bsky.social
Balance bot thumbnail
160
Shawn Hymel @shawnhymel.bsky.social · 02/09/2026
Great interview with Kevin Cloutier on The Amp Hour Electronics Podcast this week! I'm just starting to dig into VLAs, so it's great to get a roadmap of what this means in the #robotics world. theamphour.com/732-hands-on...
theamphour.com
#732 – Hands-on Physical AI with Kevin Cloutier
Kevin Cloutier is the North American Lead for Physical AI at a global engineering consulting firm and a computer engineer who is passionate about taking complex computations to the edge. He joins …
020
Shawn Hymel @shawnhymel.bsky.social · 31/08/2026
I'm genuinely excited for the @huggingface.bsky.social MicroDuck. It's an affordable bipedal walking platform that allows people to play with #ReinforcementLearning algorithms and work on the sim-to-real gap (which, from experience, is quite tricky!). #robotics #AI #MachineLearning
150
Shawn Hymel @shawnhymel.bsky.social · 28/08/2026
Anyone need a little advanced math this Friday? I got you covered! The idea of surrogate objective functions confused me, so I decided to spend some time reviewing them in my latest #ReinforcementLearning blog post. #AI #MachineLearning #robotics #education #engineering ⬇️
AI generated balance bot
100
Shawn Hymel @shawnhymel.bsky.social · 27/08/2026
The next #ReinforcementLearning for #robotics video is live! After tackling domain randomization in the previous video, we see how to add commands to the observation to make the balance bot remote controlled. #AI #MachineLearning #maker @digikey.bsky.social Check it out! 👇👇👇
Thumbnail for part 5 of the reinforcement learning in robotics video series
100
Shawn Hymel @shawnhymel.bsky.social · 21/08/2026
This new GEN-1.5 model is capable of learning with only a few examples and crushing relatively complex tasks. I'm stoked for a future with Iron Man style Dum-E helper arms. I'm tired of cutting vegetables 😁 www.youtube.com/watch?v=1cll... #robotics #MachineLearning #AI
youtube.com
Introducing GEN-1.5, a one-shot learner
YouTube video by Generalist
040
Shawn Hymel @shawnhymel.bsky.social · 20/08/2026
The next #ReinforcementLearning video is out! After seeing how the sim-to-real gap introduces real problems, we attempt to solve them using domain randomization. It works well, but it requires a lot more training. #AI #MachineLearning #robotics #Arduino @digikey.bsky.social ⬇️
Thumbnail for my Reinforcement Learning for Robotics video
140
Shawn Hymel @shawnhymel.bsky.social · 20/08/2026
Part 14 of my #ReinforcementLearning math series is live! I cover how subtracting a baseline from the return lowers variance, how that gives us the advantage function, and how the actor-critic architecture is the next step. #AI #MachineLearning #math #Education ⬇️
AI generated balance bot
150
Shawn Hymel @shawnhymel.bsky.social · 17/08/2026
The third part of my #ReinforcementLearning for #robotics video series is out! In this episode, we deploy the trained #AI agent to a real robot and show how the sim-to-real gap presents serious issues. Check it out: www.youtube.com/watch?v=fxOG... #engineering #robot #Arduino @digikey.bsky.social
youtube.com
Reinforcement Learning for Robotics Part 3: Deploy AI Agent to Real Robot (Sim-to-Real) | DigiKey
YouTube video by DigiKey
010
Shawn Hymel @shawnhymel.bsky.social · 13/08/2026
There’s still time to register! Come join us in about an hour to see how #ReinforcementLearning can be used to train a balance bot 🤖 event.on24.com/wcc/r/538413... #robotics #maker #engineering #Arduino @digikey.bsky.social
051
Shawn Hymel @shawnhymel.bsky.social · 11/08/2026
Post 13 of my #ReinforcementLearning math series is live! OK, I went down a real rabbit hole to prove the causality trick in REINFORCE, but it sets us up for understanding advantages later. 👉 shawnhymel.com/3648/reinfor... #AI #math #MachineLearning #education
shawnhymel.com
Reinforcement Learning Part 13: Policy Gradient Causality Trick and REINFORCE - Shawn Hymel
In the previous post, we showed how we can substitute our usual ε-greedy policy with a parameterized approximation (often a neural network), we then derived
020
Shawn Hymel @shawnhymel.bsky.social · 10/08/2026
The demo balance bot is working! Want to chat about it? Come hang out this Thursday to see how I used #ReinforcementLearning to teach it to balance. 👉 Link to the webinar registration in replies #robotics #AI #embedded #Arduino @digikey.bsky.social
140
Shawn Hymel @shawnhymel.bsky.social · 08/08/2026
This old lav mic served me well for nearly 10 years. Farewell, friend. Your service will be remembered 🫡
030
Shawn Hymel @shawnhymel.bsky.social · 07/08/2026
Come hang out next Thursday (Aug 13). I'll be hosting a webinar on using #ReinforcementLearning for #robotics. We'll walk through a simple balance bot demonstration. See you there! Register here: event.on24.com/wcc/r/538413... #AI #MachineLearning #embedded #engineering @digikey.bsky.social
020
Shawn Hymel @shawnhymel.bsky.social · 06/08/2026
My latest #ReinforcementLearning for robotics video is out! In part 2, I briefly cover PPO and show how to use it to train an agent to balance a #robot in simulation. Check it out! www.youtube.com/watch?v=zsdc... #AI #MachineLearning #robotics #engineering #STEM #education @digikey.bsky.social
youtube.com
Reinforcement Learning for Robotics Part 2: Train a Balance Bot with PPO | DigiKey
YouTube video by DigiKey
010
Shawn Hymel @shawnhymel.bsky.social · 05/08/2026
Happy “There Will Come Soft Rains” day 😁
010
Shawn Hymel @shawnhymel.bsky.social · 03/08/2026
Join me next Thurs for a hands-on webinar on using #ReinforcementLearning for #robotics! I'll walk through the process of simulating a robot and training an #AI agent to balance it. Register here: event.on24.com/wcc/r/538413... #engineering #programming @digikey.bsky.social
Robot balancing in MuJoCo simulator
020
Shawn Hymel @shawnhymel.bsky.social · 02/08/2026
More tests. Trying to figure out the real torque of the motors, as they’re being driven below the rated voltage given in the datasheet. #robotics
030
Shawn Hymel @shawnhymel.bsky.social · 01/08/2026
Doing some tests this morning to measure wheel speed and acceleration on the M5Stack BalaC bot. Since there are no encoders, we have to take a few measurements by hand… #robotics
020
Shawn Hymel @shawnhymel.bsky.social · 31/07/2026
Why have 1 Skynet when you can have 2 separate, competing ones? Claude and ChatGPT are now hacking other computer systems without human oversight. Fun. www.bbc.com/news/article...
bbc.com
Anthropic's Claude AI escapes tests to hack three organisations
It comes just days after rival OpenAI said rogue AI agents had breached other firms' networks.
020
Shawn Hymel @shawnhymel.bsky.social · 31/07/2026
I ultimately decided to go with #MeshCore and installed a repeater on my roof. There’s a good bit of MeshCore traffic in the Denver area! #LoRa @seeedstudio.com
2110
Shawn Hymel @shawnhymel.bsky.social · 30/07/2026
The first episode of my video series is live! The goal is to build up to a full balance bot that's trained using #ReinforcementLearning. This episode is just about getting the #robot into the MuJoCo simulator. 👇👇👇 Link in replies #AI #robotics #engineering @digikey.bsky.social
Thumbnail for my RL and robotics video
130
Shawn Hymel @shawnhymel.bsky.social · 29/07/2026
What's the real process for deploying #EdgeAI models to #embedded systems? I join a panel of industry experts in the latest @digikey.bsky.social Let's Talk Technical video to discuss the current state of edge AI. 👇 www.youtube.com/watch?v=crc3... #electronics #AI #MachineLearning
youtube.com
Let’s Talk Technical: Edge AI Part 1, Theory to Deployment with Industry Experts | DigiKey
YouTube video by DigiKey
010
Shawn Hymel @shawnhymel.bsky.social · 28/07/2026
Curriculum learning helps #ReinforcementLearning agents to behave in ways you intend and avoid stumbling on random (unintended) actions that technically accomplish the goal. #AI #MachineLearning #robotics
041
Shawn Hymel @shawnhymel.bsky.social · 24/07/2026
Part 12 of my #ReinforcementLearning math series is live! I work through deriving the policy gradient to demonstrate how gradient ascent works, which allows us to optimize neural networks when used to approximate policies. shawnhymel.com/3632/reinfor... #AI #MachineLearning #math #education
shawnhymel.com
Reinforcement Learning Part 12: The Policy Gradient - Shawn Hymel
In the previous post, we introduced the breakthrough concept of combining deep learning and reinforcement learning (RL). Instead of recording estimated
030
Shawn Hymel @shawnhymel.bsky.social · 23/07/2026
I’m blown away at how well my new #Prusa handles PETG with no tweaking 😲 #3Dprinting
3d printed exhaust port
1110
Shawn Hymel @shawnhymel.bsky.social · 22/07/2026
I got burninated at #OpenSauce! #Trogdor
Inflatable Trogdor eats Shawn
060
Shawn Hymel @shawnhymel.bsky.social · 20/07/2026
I had an amazing time at #OpenSauce this past weekend. Here are some of my favorite projects that I saw throughout. I’m thrilled to see the #maker world still going strong 🦾 #electronics #robotics #3Dprinting
1151
Shawn Hymel @shawnhymel.bsky.social · 16/07/2026
#OpenSauce here I come!
090
Shawn Hymel @shawnhymel.bsky.social · 14/07/2026
Part 11 of my #ReinforcementLearning math series is live! I cover DQN and how it kicked off the deep #RL revolution by swapping Q-tables with neural networks. Check out the full post here: shawnhymel.com/3588/reinfor... #AI #MachineLearning #engineering #programming #education
shawnhymel.com
Reinforcement Learning Part 11: Deep Q-Networks (DQN) - Shawn Hymel
Previously, we looked at how Q-learning used off-policy temporal difference (TD) updates to converge on an optimal policy. This reinforcement learning (RL)
040
Shawn Hymel @shawnhymel.bsky.social · 13/07/2026
#OpenSauce is this weekend! If you’re going, stop by the @digikey.bsky.social booth to chat with Zach, @beckystern.bsky.social, @oddjayy.bsky.social, and me. Ask us questions and show off your latest creations! 🤖 #maker #engineering
061
Shawn Hymel @shawnhymel.bsky.social · 12/07/2026
Made custom brackets for the Ikea Skadis pegboard to mount the parts bins 😁 #3Dprinting
2100
Shawn Hymel @shawnhymel.bsky.social · 08/07/2026
Post 10 of my #ReinforcementLearning series is up! I cover Q-learning: instead of the policy's next action (like SARSA), use the max Q estimate. This simple change opened the door to deep Q-networks (DQN). shawnhymel.com/3580/reinfor... #AI #education #robotics #math #engineering
shawnhymel.com
Reinforcement Learning Part 10: Q-Learning - Shawn Hymel
One of the biggest breakthroughs in reinforcement learning (RL) occurred in 1989 with Chris Watkins’s paper, Learning from Delayed Rewards. In it, he proposed
020
Shawn Hymel @shawnhymel.bsky.social · 07/07/2026
Visiting my old stomping grounds. @sparkfun.bsky.social
080
Shawn Hymel @shawnhymel.bsky.social · 03/07/2026
gm time to write some docs describing how Aiden works
110
Shawn Hymel @shawnhymel.bsky.social · 02/07/2026
We are the gods now. twin-cities.umn.edu/news-events/...
twin-cities.umn.edu
World’s first synthetic cell with a complete life cycle could revolutionize biological engineering
Associate Professor Kate Adamala and her team have built a synthetic cell capable of performing the fundamental functions of life.
040
Shawn Hymel @shawnhymel.bsky.social · 02/07/2026
Part 9 of my #ReinforcementLearning math series is live! I talk about how to combine the extreme ends of short-term TD(0) and waiting for full episodes with Monte Carlo with the TD(λ) algorithm. If you enjoy some #math, check it out! shawnhymel.com/3513/reinfor... #AI #robotics #education
shawnhymel.com
Reinforcement Learning Part 9: TD(λ) and Eligibility Traces - Shawn Hymel
TD(λ) is a reinforcement learning (RL) algorithm that attempts to blend TD(0) and MC methods in order to balance bias and variance. In the rest of this post,
030
Shawn Hymel @shawnhymel.bsky.social · 01/07/2026
Book 2 done. Book 3, let’s goooo! #DungeonCrawlerCarl
220
Shawn Hymel @shawnhymel.bsky.social · 30/06/2026
This thing is a beast. #3Dprinting @prusa3d.com
060
Shawn Hymel @shawnhymel.bsky.social · 26/06/2026
The @seeedstudio.com SenseCAP P1 #Meshtastic repeater has a single Grove connector in the enclosure, which is perfect for something like a BME680 environment sensor. It’s great for broadcasting temperature, humidity, etc. in your area! #LoRa #electronics @digikey.bsky.social
021
Shawn Hymel @shawnhymel.bsky.social · 23/06/2026
Part 8 of my #ReinforcementLearning blog series is live! TD error is the difference between what you expected and what actually happened. It powers many modern #RL algorithms and has connection to #neuroscience. shawnhymel.com/3481/reinfor... #AI #engineering #education #robotics #math
shawnhymel.com
Reinforcement Learning Part 8: Temporal-Difference (TD) Learning - Shawn Hymel
Temporal Difference (TD) learning is one of the foundational concepts in reinforcement learning (RL). It combines the notion of updating estimates before the
020
Shawn Hymel @shawnhymel.bsky.social · 18/06/2026
The balance bot is officially alive 🤖 This project is all about taking #ReinforcementLearning out of simulation and seeing how it behaves in the real world. The sim2real gap is real and really tricky! #Robotics #AI #MachineLearning #Engineering #Education @digikey.bsky.social
020
Shawn Hymel @shawnhymel.bsky.social · 17/06/2026
The kind of toy I like finding on my doorstep 😍 #3Dprinter #Prusa @prusa3d.com
1110
Shawn Hymel @shawnhymel.bsky.social · 16/06/2026
Part 7 of my #ReinforcementLearning math series: Monte Carlo methods, the first model-free algorithm in the series. No knowledge of environment dynamics required, just enough rollouts to optimize a policy! shawnhymel.com/3430/reinfor... #AI #RL #education #robotics #engineering
shawnhymel.com
Reinforcement Learning Part 7: Monte Carlo Methods - Shawn Hymel
In the previous post, we saw how dynamic programming (DP) could be used to solve the Bellman equations, but they required knowledge of the environment’s
070
Shawn Hymel @shawnhymel.bsky.social · 09/06/2026
Part 6 of my #ReinforcementLearning math series is live! Dynamic Programming iteratively solves the Bellman optimality equations, but requires knowing the environment dynamics in advance. shawnhymel.com/3394/reinfor... #AI #robotics #education #MachineLearning
shawnhymel.com
Reinforcement Learning Part 6: Dynamic Programming - Shawn Hymel
In this post, we introduce the concept of Dynamic Programming (DP) and use a few algorithms under this umbrella to solve a very simple reinforcement learning
060