Recent searches

No recent searches

Search options

Only available when logged in.
med-mastodon.com is one of the many independent Mastodon servers you can use to participate in the fediverse.
Medical community on Mastodon

Administered by:

Server stats:

351
active users

med-mastodon.com: AboutProfiles directoryPrivacy policy

Mastodon: AboutGet the appKeyboard shortcutsView source codev4.4.2

#TinyML

0 posts0 participants0 posts today
Laura Sophia

My old introduction was very outdated, so it's time to reintroduce myself:
#introduction

Hi 👋, I’m Laura.

I am a transfeminine person, somewhat in the middle of my transition. 🏳️‍⚧️ #trans #transbubble

A major part of my time I spend as a Postdoc in computer science, working on embedded AI and low-power IoT communication. #cs #TinyML #IoT #academia #science

Outside of work, I am active in the local #queer center (board member, GER: Vorstand), I enjoy playing board games, and I listen to too many #podcasts.

Alasdair Allan

By me for @hackster_io, "Benchmarking TensorFlow and TensorFlow Lite on Raspberry Pi 5." The big take away from these new benchmarks is that the Raspberry Pi 5 has similar performance when using TensorFlow Lite to the Coral TPU, displaying essentially the same inferencing speed as Google's accelerator hardware. #ML #TinyML #AI #TensorFlow #RaspberryPi #CoralTPU hackster.io/news/benchmarking-

Hackster.io · Benchmarking TensorFlow and TensorFlow Lite on Raspberry Pi 5By Alasdair Allan
Alasdair Allan

Here we go! Depends how much of a purist you are whether you regard these as #TinyML or not. But you have to admit, it's #LLM on a comparatively tight resource budget. arxiv.org/abs/2312.11514

arXiv.orgLLM in a flash: Efficient Large Language Model Inference with Limited MemoryLarge language models (LLMs) are central to modern natural language processing, delivering exceptional performance in various tasks. However, their substantial computational and memory requirements present challenges, especially for devices with limited DRAM capacity. This paper tackles the challenge of efficiently running LLMs that exceed the available DRAM capacity by storing the model parameters in flash memory, but bringing them on demand to DRAM. Our method involves constructing an inference cost model that takes into account the characteristics of flash memory, guiding us to optimize in two critical areas: reducing the volume of data transferred from flash and reading data in larger, more contiguous chunks. Within this hardware-informed framework, we introduce two principal techniques. First, "windowing" strategically reduces data transfer by reusing previously activated neurons, and second, "row-column bundling", tailored to the sequential data access strengths of flash memory, increases the size of data chunks read from flash memory. These methods collectively enable running models up to twice the size of the available DRAM, with a 4-5x and 20-25x increase in inference speed compared to naive loading approaches in CPU and GPU, respectively. Our integration of sparsity awareness, context-adaptive loading, and a hardware-oriented design paves the way for effective inference of LLMs on devices with limited memory.
TrendingLive feeds
About