r/Python • • Dec 13 '19

Python + Quantum Research = Art + Saving 1000s hours of CPU time

1.2k Upvotes

29 comments sorted by

97

u/basnijholt Dec 13 '19 edited Dec 13 '19

Adaptive: parallel active learning of mathematical functions

This is a quantum physics calculation in progress.

When evaluating a function numerically we would like to sample it more densely in the interesting regions instead of evaluating it on a manually-defined homogeneous grid.  Together with my colleagues, we wrote an open-source Python software package Adaptive that evaluates the function at the optimal points by analyzing existing data and planning ahead on the fly. With a few lines of code you define your goal, evaluate functions on a computing cluster, and live-plot the data. It performs averaging of stochastic functions, interpolation of vector-valued one and two-dimensional functions, and one-dimensional integration.

This figure is from one of my recent papers Spin-Orbit Protection of Induced Superconductivity in Majorana Nanowires.

What you see here is how the calculation progresses in time. You see that as time progresses it starts to find the narrow lines (which have a physical meaning). For aesthetic purposes, I change the colormap slowly which is meaningless but pretty. The vertices (corners) of the white triangles on top of the plot represent the points are which the underlying function is evaluated. The >1000 years of computation time is based on a comparison between a homogeneous grid. If we take the size of the smallest triangle, the computation time with adaptive (2 weeks on 300 cores), and assume that all the features larger than the smallest triangles have been resolved, then the calculation would have taken 3134 years.

You can recreate this video by using this script and this data. The entire source code for generating the data is here (be warned, you probably need a supercomputer).

See (star) ⭐️ the repo on github.com/python-adaptive/adaptive and the documentation on adaptive.readthedocs.io.

Try pip install adaptive[notebook] or conda install adaptive :-)

You can also run Adaptive code live in Binder here!

P.S. adaptive has already been used in several other scientific publications, see the gallery!

31

u/discobrisco Dec 13 '19

I hope you share this outside of just this sub, awesome work.

19

u/basnijholt Dec 13 '19

Thanks! Any recommendations where to cross-post this? All of the cool work is in the Python package by the way :)

25

u/inspectorG4dget Dec 14 '19

/r/python may be a good start

FWIW, I'm going to try to find a way to use this for hyperparamaeter optimization in machine learning (tuning machine learning algorithm parameters). So maybe a machine learning sub would be interested in this as well?

53

u/ItsBarney01 Dec 14 '19

This is r/python mate

24

u/inspectorG4dget Dec 14 '19

well, sh!t

I think I need to pack up and go home, now

25

u/BlueTeeJay Dec 14 '19

So deep in imports you forgot it was python.

8

u/inspectorG4dget Dec 14 '19

If only someone could write a contextlib for my brain...

3

u/DatBoi_BP Dec 14 '19

Where did you think you were, r/compsci?

5

u/ItsBarney01 Dec 14 '19

You might be able to try r/simulated While it's not the normal content you would get there, it IS a simulation and it's visually pleasing so might be worth a post :D

2

u/Cynox Dec 14 '19

0

u/[deleted] Dec 14 '19

[deleted]

3

u/NoelFromBandOsmosis Dec 14 '19

Here’s a sneak peek of the worst accounts on Reddit using the worst accounts of all time!

#1: u/sneakpeekbot | 701,180 karma
#2: u/The-Worst-Bot | -58 karma
#3: u/CommonMisspellingBot | 574,571 karma

I’m not a bot, I am a human who hates those bots | Downvote and I probably won’t delete the comment | Don’t contact me

3

u/Bob312312 Dec 14 '19

Hi did you write this? - It is super cool!

I was very interested in this project a while back. But I have one main question. Is there a way to use it where you pick multiple points as any given time?

I was thinking of a case where you might use an HPC. Say I have 32 cores and can calculate 32 points at a time. Can I chose the best 32, run the calculations and then pick the next best 32 points once all of the calculations have returned?

best bob

27

u/sp8ial Dec 14 '19

Sexy lady

6

u/DatBoi_BP Dec 14 '19

Whopa Gangnam Style

2

u/whudaboutit Dec 14 '19

I'm with you. I think I spent too much time on those other subreddits.

20

u/BarryScott2019 Dec 14 '19

This is the same shit you see when you rub your eyes too much. I like.

11

u/[deleted] Dec 14 '19

Soo computers loves tits?

5

u/Rocket089 Dec 14 '19

I’m not the only one who thought of an hour glass figure with a great rack?!

YESSS 💪🏼

1

u/MagicalTacos Dec 14 '19

Who doesn't

1

u/[deleted] Dec 14 '19

Is this a band structure calculation?

2

u/basnijholt Dec 14 '19

It is the conductance through nanowire.

But a band structure is something that we typically calculate as well.

1

u/[deleted] Dec 14 '19

Spintronics?

2

u/zQuantz Dec 13 '19

This looks amazing!

2

u/7JKS Dec 14 '19

2

u/VredditDownloader Dec 14 '19

beep. boop. I'm a bot that provides downloadable video links!

I also work with links sent by PM


Info | Support me ❤ | Github

0

u/QuietRains813 Dec 14 '19

Why CPU time, and not GPU time?

1

u/ErikRK Numba fan Dec 15 '19

Prob because GPU programming is much harder with adaptive algoritms