Research Interests
Publications
Awards & Honors
Research Internships
Service
I am a PhD student at the University of Washington advised by Luke Zettlemoyer working on representation learning, and neuro-inspired and hardware optimized deep learning. Previously I interned at the UCL Machine Reading Group where I was advised by Sebastian Riedel working on information retrieval and link prediction in knowledge graphs. I did my master in computer science at the University of Lugano, where I was advised by Fabio Crestani. During my master, I did two research internship at Microsoft under Chris Basoglu, where I worked speech recognition and memory efficient deep learning algorithms. I did a bachelor in applied mathematics with The Open University while working as a software engineer in the automation industry.
In 2013, I also took part in Kaggle competitions where I peaked at world rank 63 (top 0.22%).
Feel free to contact me at firstname.lastname@gmail.com; if you have questions regarding deep learning, I prefer that you post your questions as comments on one of my blog posts (or on this page if it does not fit to any blog post); this way all people can profit from your questions and my answers.
Research Interests
My main research thesis is that computational efficient methods will accelerate and enable progress in and understanding of deep learning. In particular, I am interested in:
- Sparse learning
- Representation learning
- Understanding deep learning
- Neuro-inspired deep learning
- Hardware optimized deep learning
Publications
2019
Sparse Networks from Scratch: Faster Training without Losing Performance. Tim Dettmers, Luke Zettlemoyer. under review. [arXiv] [bib] [code] [blog post] [Q&A]
2018
Jack the Reader – A Machine Reading Framework, Dirk Weissenborn, Pasquale Minervini, Tim Dettmers, Isabelle Augenstein, Johannes Welbl, Tim Rocktäschel, Matko Bošnjak, Jeff Mitchell, Thomas Demeester, Pontus Stenetorp, Sebastian Riedel. ACL, 2018. [arXiv] [bib] [code]
Convolutional 2D Knowledge Graph Embeddings, Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, Sebastian Riedel. AAAI, 2018 (acceptance rate 25%). [arXiv] [bib] [code] [data] [Q&A]
2016
8-Bit Approximations for Parallelism in Deep Learning, Tim Dettmers, 4th International Conference on Learning Representations (ICLR), 2016 (conference track, acceptance rate 25%). [arXiv] [bib] [code] [data]
Computational and Parallel Deep Learning Performance Benchmarks for the Xeon Phi, Tim Dettmers, Hanieh Soleimani, technical report, 2016.
Awards & Honors
2018/2019 Jeff Dean – Heidi Hopper Endowed Regental Fellowship
2016/2017 Google Scholarship for Students with Disabilities
2016 ICRL 2016 Travel Award
2011/2012 Best regional graduate: Mathematical-technical Software Developer
Research internships
2017-09 – Now UCL Machine Reading Group, London
2017-06 – 2017-09 Microsoft Research, Redmond
2017-01 – 2017-06 UCL Machine Reading Group, London
2016-06 – 2016-09 Microsoft Research, Redmond
Service
I am reviewing for Knowledge and Information Systems.
Are you building your own ml machines or buying gaming machines with Nvidia 980 gpu’s? What is a good desktop to buy for ml? I do not really want to put together my own computer.
It is always significantly cheaper to build one’s own machine and you can customize the parts better. Gaming machine often have some parts that are better than needed and some important parts which worse than needed. However, for single GPU systems overall performance is determined by the GPU (about 90%) and not by the surrounding hardware (about 10%); so any gaming rig with a GTX 980 or any other good deep learning card will be all that you need – all choices are good.
Can i use Matlab for deep learning to make a project on speech recognition? I need to add this program in a system in a bank for voice recognition.
A quick google search yielded these matlab deep learning libraries:
https://github.com/huashiyiqike/LSTM-MATLAB
http://www.vlfeat.org/matconvnet/
Hope that helps.
What reading list (blogs/online courses/books) would you suggest for someone with an economics background and wished to become competitive at Kaggle?
Keep up the good work!
The Cousera ML course by Andrew Ng is excellent and will get you acquainted with all different basic machine learning algorithms. With an economics background you may already know some of these algorithms, but a refresher might be nice.
Once you done that the main goal would be to become good at feature engineering. This is a very difficult task, and there is little information available – you will mainly learn through experience. The most important resource will be Kaggle forum threads of past competitions where the winners explain their methods; try to go through all the old competitions and understand the methodology of the winners – you will learn a lot from this! Also look at “Beating the benchmark threads” for each competition. There you will usually be able to learn easy basics to get started on problem.
After this you will mainly learn through experience: I would use python and the scikit-learn (sklearn) module and start to do as many competitions as you can. Try to apply what the winners mentioned in their forum posts. From here it is practice, practice, practice: Initially you will have little success, but do not be discouraged by this, this is normal; it will take a few months until you will score well, and about 1-2 years until you will perform at masters level; 3-7 years and you will perform at world class level.
Hope this helps – good luck!
Very much appreciated Tim. its going to be a long journey but I look forward to completing it. See you at Kaggle!
Hi. Thank you very much for awesome posts about hardwares. It helped me a lot. By the way, I am just curious – what library do you use for your deep learning networks?
Thanks for your feedback. Currently, I am developing my own deep learning library from scratch, which will feature automatic GPU computation for any algorithm and numpy interface. Besides that I found Torch7 to be best (among theano/pylearn2 and caffe): It is easy to use, features many highly optimized routines that other libraries do not offer and it is easy to extend.
Hi, looking at this, is torch7 still relevant for DL with around 10gb to 30gb data? It is easier to install torch7 compared to Theano, but, my question is, how about the Gpu utilization?
P/s thanks for that extensive guide on choosing the right gpu.
Torch7 and theano are good and should offer about the same GPU utilization.
Hey! Just saw your blog. Nice endeavor. I’m an undergrad myself as well. I saw your article on Hardware for Deep Learning. I had been thinking of building a machine for my research. Can you shell out some details about your system?
I currently have 4x GTX Titan, i7 4820K (4core, 8 threads), PCIe 3.0 CPU, 32GB RAM and a SSD + 1TB HD.
Your website is amazing. Thank you so much for putting these together.
What is the motherboard you are using?
I am putting together the following – what do you feel? If I use AMD CPU with PCIe 2.0 then the cost could come down by 400 $. Would I be sacrificing lot by using a PCIe 2.0 ?
Thank you,
Deepak
Component Selection Price
CPU
Intel Core i7-4820K 3.7GHz Quad-Core Processor
$319.99 Buy
CPU Cooler
RAIJINTEK AIDOS BLACK 48.6 CFM Sleeve Bearing CPU Cooler
$24.21 Buy
Motherboard
Asus Rampage IV Gene Micro ATX LGA2011 Motherboard
From parametric selection (show)
$229.99 Buy
Memory
G.Skill Ripjaws X Series 8GB (2 x 4GB) DDR3-1600 Memory
$59.98 Buy
Storage
Western Digital Caviar Blue 1TB 3.5″ 7200RPM Internal Hard Drive
$51.49 Buy
Video Card
EVGA GeForce GTX 960 4GB SuperSC ACX 2.0+ Video Card
$238.98 Buy
Case
Corsair 100R ATX Mid Tower Case
$47.99 Buy
Power Supply
Corsair CX 500W 80+ Bronze Certified Semi-Modular ATX Power Supply
$66.98 Buy
Total: $1039.61
I have an Asus Rampage and a Gigabyte motherboard; for most people the Asus Rampage did work fine, but for me it would not work with 4 GPUs. So I switched to my Gigabyte motherboard which just works fine with 4 GPUs. Generally there is not much wrong you can do with a motherboard — in my case it was just bad luck with the Asus Rampage.
I have no data from AMD CPUs, but because CPUs are not that important, I do not think you can go wrong with that. If you roll with PCIe 2.0, then you really should only work with one GPU (0-10 % performance decrease); otherwise, if you plan to have multiple GPUs in the future try to get a PCIe 3.0 CPU and motherboard (4 GPUs about 40% performance decrease for PCIe 2.0). AMD vs. Intel should be at most 2-5 % performance decrease.
An undergrad?
Really surprised me, your articles are well organized that I think you might be an young professor or engineer!
I am new to deep learning, and I find many preliminary concept from your blogs. That helps me a lot, since my mentor just sends me a series of papers, I am confused by the jargon.
Thank you! It is quite ironical, because I flunked out of high school due to my writing. I am a dyslexic and while I have a terrible time to write with pen-and-paper I am much better at this when I can use a computer. This might seem quite paradoxical, but this is actually quite common for dyslexics: Dyslexics can often write well, but they will need help with the writing to just do that — I am so grateful that I live in a time where computers exist that can help me with that.
I have to agree with jokeren! Very impressive blog and great subject area expertise from an undergrad. Look forward to following your blog as I begin to learn about this field! Do you have a space where I can ask you or other experts questions?
Thank you! It is best to ask questions in the form of a comment under the most appropriate blog post or here for anything else, that way the answers and questions will be visible to anyone else.
I am extremely impressed with your blog posts. I am also amazed to find that you are dyslexic like myself. Do you also find it much easier to read from a computer screen than paper text? I have noticed from mid-high school that it was much easier for me to comprehend digitized text over traditional ink and paper. Please keep up the amazing work, and I hope one day to be reading about advancements that you have provided to the AI world.
Thank you for the comment! Indeed it is much easier to read from a monitor. Computers are also such a blessing to work with! I am so glad that the computer can check my grammar/spelling and it is so easy to move and structure text and sentences. We dyslexics are lucky to live in such a great time — cheers!
Thank you for an interesting blog. Would be great to meet, I am a few hundred Km North of you!
Unfortunately, I will move to Switzerland soon where I will study at the University of Lugano. Currently, I am quite busy with organizing my move. So if you want to discuss/talk about things the best way to reach me is via email. If you have some questions you are always free to ask them here, so that other people will be able to see the Q&A here.
Thanks for your excellent blogs. What’s your email? I want to discuss more about deep learning with you.
My email is firstname.lastname@gmail.com. If you have specific questions I prefer if you ask those questions as a comment on a specific blog post. That way the question/answer will be accessible for everyone.
Hi Tim,
Often I read your very useful articles. I just want to know about how can I connect an external GPU (telsa K40) with my laptop ( Intel core i5 2.5 GHz with 8GB RAM). I want to run some deep learning experiments using GPU ( caffe library).
Is there any dock type system to correct external GPU with laptop? If you provide me some useful link to study, would be great.
Many thanks.
Ranju Mandal
My name is Canaan Tinotenda Madongo and am a 2nd year masters student at Beihang University in Beijing, China. Am doing my research on Person Re-identification, were am also supposed to have an understanding of Deep learning and its application to Person Re-identification. Am asking for your help on the above mentioned, regarding how best i can understand Deep learning and its applicablility to my research topic. Your assistance will be greatly appreciated as am new to this interesting topic of Deep Learning.
I am planning to write two blog posts on the topic of how to learn deep learning, but currently I have no timeline for that. I do not have the resources to help you personally, and it is difficult to give you guidance from the info that you provided. You can read my Deep Learning in a Nutshell series to get started, and after that you might read some papers, take some courses, or even jump right into software frameworks and learn how to train networks on small dataset.
Thank you so much Tim, I really appreciate it…
Hi Tim,
I am using GTX 1070 and have met some trouble installing Ubuntu on my Windows 10 machine, the problem is described here.
http://askubuntu.com/questions/812985/how-do-i-deal-with-the-input-signal-out-of-range-problem-while-installing-ubun
I would be more than appreciated if can look in to the problem and love to help! I have been haunted by this for days.
Best,
Tony
I never had such an error. Do you install ubuntu via the Windows installer? Also this link might be relevant.
Hey,
I would like to buy Nvidia GTX 1060 GPU but I am stuck on choosing one from these brands. Could you help me with the best one from below:
1. Zotac Amp Edition
2. Zotac Mini GTX 1060
3. MSI Armor
4. MSI Gaming X
Thanks and Regards
Harsha
The brands all have the same chip so it is all about the cooling that they provide. I do not know much about fan design, but you might find some benchmarks for different temperatures for each brand in reviews for games.
Hey Tim,
Thanks for coming back. From here , http://gadgets.ndtv.com/laptops/reviews/msi-geforce-gtx-1060-gaming-x-and-zotac-geforce-gtx-1060-amp-edition-review-862713.
Msi is having good performance . However, it says the size could be a problem. Could you help me with any good CPU where I could install this MSi GPU or the Zotac’s GPU?
I am really a tyro with GPU’s, I need some guidance before I procure them.
The size should be fine. The GTX 1060 is a relatively short GPU even the “longer” version by Zotac should be no problem. For the CPU any CPU will do if you are planning on using just one GPU. For multiple GPUs it is important that your CPU supports at least 32 PCIe lanes. You can find this number in the specifications or in the case of intel on ark.intel.com next to the label “Max # of PCI Express Lanes” field.
Hi,
I am currently using DL CNN to develop image based search solution for an ecommerce company. They have about 500 categories, main problem is there are many categories where the variance within categories is very high i.e. a single sports category will have hockey stick and then box of hockey sticks where whole hockey stick is not visible, there are many categories like these. Any suggestions you have for me.
Not much that you can do there. This is often so with real world data. You can either get more data or try to get cleaner data (preferably both). Academic data sets are usually also cleaned up in this more manual and tedious way (or rather with Amazon mechanical turk and the like).
Hi
Really appreciate your sharing. I’ve also just started to learn DL. This is really helps.
I’ve done a simple test of object detection on Linux. Now I’d like to try to port it to mobile devices, like a Andriod phone or Raspberry Pi. I mean, I’d deploy a CNN network on mobile devices to implement object detection application. I was wondering could you kindly provide some tips to choose hardwares on mobile devices?
About the CNN model, memory consumption during deployment is about 500MB per image, and computation complexity of one forward pass is about 8.35E+10 FLOPs.
It is difficult to make explicit choices for the mobile phone hardware, because often you want all mobile phone users to use your software. So the most general option would be to run it on the mobile phone CPU. This of course requires some pretty sophisticated shrinking and optimization of the model. This is not yet in reach for most users of deep learning, but for small applications there is TensorFlow and now Caffe2, but both options still seem quite experimental.
If you want to aim at specific phones, then the best ones should be ones that have the Tegra K1 chip. NVIDIA’s cuDNN works on these ships and provides quite some computing power. Memory might still be in issue here though, but you could work on your own framework to use it on such devices. However, keep in mind that your software might become obsolete quickly as software firms working internally on android frameworks might open source them at any time.
Hi Tim,
Cool looking graphs of the gridworld in your post on NVidia: https://devblogs.nvidia.com/parallelforall/deep-learning-nutshell-reinforcement-learning/
I want to use similar ones in my talks… Did you use some existing visualizer/framework or wrote the code yourself? Could you refer/share?
Thanks very much in advance!
Hi Denis,
I used Inkscape for this. I only had worked once before with Inkscape, so it still took some time to produce these figures, but I found the entire experience of designing figures quite good. I think with more experience it will be quite painless and fast to design nice figures. I would recommend Inkscape.
Ah, I see, so you didn’t run some RL learning algo to generate the plots. I also resort to Inkscape for my lectures when have no time to write code for experiments 🙂 But for RL slides – want to generate plots from real experiments to show various aspects of learning. Tried to find a simple-to-use gridworld visualizer and didn’t find one… Ended up writing one myself.
Anyway, thanks for the recommendation!
Denis.
Hi Tim,
I’m building my own Deep learning Rig!
From your experience which of the graphics cards do you recommend.
Pascal Titan X or Pascal GTX 1080 Ti?
From the Ti announcement, it seem 1080 Ti is even faster than Titan X. Does that performance edge carries over to deep learning?
Thanks in Advance,
Venkat
If you can wait for the GTX 1080 Ti, definitely go for the GTX 1080 Ti. It just gives you much more bang for the buck and the 1GB difference is negligible.
Hi Tim,
Thanks for the clarification. I already have a couple of them on pre-order. I hope it would’t be too late.
Best,
Venkat
Hello Tim,
When you’re running you’re neural network, in which proportion the DDR4 SDRAM is important with respect to the GDDR ? What should be the minimum amount of it ?
Which models of CPU support at least 32 PCIe lanes (SLI ready) ? It seems to be really expensive one.
Thanks in Advance,
Gaylou
Having 1:1 for CPU:GPU memory is often a good choice. Depending on your use-case and software framework that you use this will differ, but generally you will be okay with 1:1. If you have more than 1 GPU, you can often relax this to 1:2 in terms of CPU:GPU memory.
The lanes are not that important, only if you plan to do a lot of parallelization across GPUs. Often you do not want to do that anyways, because it is messy and requires special code in most cases. So I would go with a cheaper setup with less lanes and just use no parallelism across GPUs.
Hi Tim,
I am new to deep learning but serious about it. Do you think, using Ubuntu on the following gaming machine would be a good starting point? I will be coding models using Python.
https://www.razerzone.com/gb-en/gaming-systems/razer-blade
Theoretically it should work and the laptop seems like a good choice, practically probably not so much. The problem with laptops can be that their hardware is often not well supported by Ubuntu. So before buying a laptop you should make sure that you can actually run its hardware with Ubuntu. Usually, Lenovo laptops provide the best support for Linux systems, but even here very new Lenovo laptops do usually not have full Ubuntu (or Linux) support. Often you can get it working with some hacks here and there such as custom kernels, but this is difficult, especially if you do not have the experience. In some cases kernel updates will break NVIDIA drivers and thus render your laptop unusable for CUDA / deep learning.
So again, make sure your hardware is supported. If it is supported the laptop that you are linking is a good choice.
Thank you so much Tim.
I love your blog, it’s been extremely useful for building my own DL
setup. Right now I’m undertaking a large scale project, and as helpful
and informative as your blog posts are, I cannot piece together all the
necessary information from them and hence I’m asking you for help.
The aim is to have multiple nets running in one server. I think this is
possible by allocating different GPUs inside the server to separate
tasks, for example GPU0 for NetA and GPU 12 for NetB. I’m planning to
build a server with multiple GPUs inside it for this task (quite
obviously). Currently what I want to do is have one huge server with
lots of compute capability, and have all the developers sitting on
separate laptops or something similar developing the code. Later when
the developer is done he will copy the python (or whatever) files onto
the server via just a plain old USB (still to be finalized) and then run
the python files using the terminal on the server itself. Will this
work? Will this cause too much of an inconvenience to the devs? Is this
a better alternative to Virtual Machines?
I have a few more questions to bug you with :P. Could you please give me an email address? I wish to discuss this in more detail with you. I actually tried to mail firstname.lastname@gmail.com…I’m not very sure what I was expecting (it told it was an invalid address).
Thanks
This will work very fine and this seems to be a standard workflow (expect developers will copy the files using the network and not a usb). I would not virtualize the nodes since that will often cause problems with the GPUs. In general, if you have machines with many GPUs and you give developer ssh access it will be more than fine and they will be able to do their job. You can restrict access so that developers can only change local files; for that to work of course, you will need to setup all the tools that the developers need beforehand, or force them to use virtual environments as offered by local Anaconda installations.
The email should be correct; tim.dettmers@gmail.com.
Thanks a bunch! This makes my life so much easier. I’m planning to setup 6 machines with 20 GPUs each, linked together via InfiniBand so it becomes as good as one machine with 120 GPUs. Afterwards, is it as easy as installing Ubuntu 14.04 LTS on the head node and opening up different terminals and choosing different GPUs? I heard something about slurm and how it is needed in HPC clusters, what is the deal with that?
Thank you once again
Is firstname.lastname@gmail.com your current email address?
Love your blog posts!
Yes, it is tim.dettmers@gmail.com.
Hello Tim,
I’m new to deep learning and I’m thinking about building a system. What do you think of the new x399 platform and thread ripper? I wanted to start off with two Titans and upgrade to four later down the line but I wasn’t sure if I could take full advantage of the Titans with the x399 platform. Also if you don’t think x399 or AMD is a good choice which boards would you recommend?
It is a bit too early to tell how good the x399 boards / CPUs will be. I think 64 lanes are overkill for any rig that has less than 4 GPUs. The extra cores are not really needed for deep learning as is the DDR4 RAM. Overall it looks really fancy and expensive with no real payoff. The x399 platform might be more suitable for other domains, but for deep learning, it does not give any big benefit. I personally would probably go with a used DDR3 system with 36+ lanes CPU / board and some GTX 1070.
Hi Tim. Thanks for your blog. Posting this as of July 2017. I’m a doctor/researcher interested in training convolutional neural networks (densenets, resnets, inception nets) using tensorflow, keras, & python. My current system is old (2GB GPU, 8GB RAM) so, I need a new build.
I’ve settled on the ASUS X99 E 10G WS motherboard which has 40 PCIe lanes and multiple PCIe slots. CPU a 6 core i7 6850K, 64GB of fast DDR4 ram, with a 1 TB SSD and larger 8-10TB Hard drives. Only ONE GPU card for now.
Decision is between the 12GB Titan Xp (April 2017, $1500) and 11GB GTX 1080 Ti (2017 release, $750). With the new Volta P100 16GB coming out soon (volta’s 5120 cores vs pascal’s 3584), I’m afraid of serious buyer’s remorse with the Titan Xp. (I really want that extra 4GB!)
I think the answer is to get the 11GB GTX 1080Ti while I wait for the Volta P100 PCIe to be released, and then sell the 1080Ti’s & upgrade when it is out. I am concerned Titan Xp will take a huge hit in resale price when volta comes on market.
Questions:
A) Do you agree?
B) Can I mix/match the 1080Ti with 1-2 Volta 100 later, will it cause incompatibility problems or slow my training?
C) Heard of any problems with the 1080Ti & CUDA/Cudnn or other DL incompatibilities? Or works like the Titan Xp?
Thanks for your thoughts.
A) I would not buy a Volta V100 in any case, it will be very cost inefficient at about $4k-$6k a piece. There will be a commercial variant however in 2018 Q1/Q2.
B) I would do exactly this. You will not be able to parallelize across all GPUs, but you can still train models on the GTX 1080 Ti and parallelize across the GTX 1180s.
C) The GTX 1080 Ti is good. No problems for any deep learning application / software
Small typo in the year of your internship
“Research internships
2016-06 – Now Microsoft Research, Redmond”
Thanks for catching that!
Hi Tim,
do you provide 1 to 1 lectures ? can i have your email for further discussion ?
Thank you
Hi Tim,
Can we use TFLearn API for distributed training of the DNN over cluster of GPU or over Spark?
Thanks!
TensorFlow, in theory, supports distributed training. But I have not yet seen any successful implementation or usage of that outside of Google. So you can use it, but probably you do not want to use it. If you need parallel training over GPUs via Spark you can look at CNTK and their distributed computing framework. It is well designed for such as task, although I have not seen it being used outside of Microsoft either.
Hi, Tim!
I am new for machine learning.
Which laptop you will suggest to buy for beginning for running MATLAB, python, R, excel, tensor flow and further machine learning? Am I right for looking for laptop of less than 900 USD cost with DELL Vostro 5468, DELL Vostro 5568, ASUS S406UA, ASUS S510UQ, HP 15-bs563ur, Acer Swift 3 SF315-51? Which of them you would advice me to buy?
Should I buy with intel graphics 620 + amd radeon 530 or intel graphics 620 + nvidia ge force 940 mx? Is i5 enough and ssd of 256 gb for beginning my journey?
Try to buy a laptop with an NVIDIA GPU, that should get you started for TensorFlow and machine learning. To just try things out your GPU just needs about 1-2GB of RAM. If you want to be a bit more serious about it try to find an NVIDIA GPU with 4-6GB or more RAM. The rest does not matter that much.
Tim-
I’m an avid fan of your blogs, and the mix of practical advice and theory. We’re working on pedestrian avoidance using in-car mini-ITX servers. With Texas Inst c66x devices we can fit 64 c66x cores into a 4″ x 9 ” x 9″ box using half-size PCIe cards, along with a quad-core Atom, all under 150 W. c66x devices are extremely energy efficient at 8/16-bit convolutions with 32-bit acc, also they are C/C++ programmable so we have lots of options for running the most recent compressed models, and running different ones concurrently.
The problem with using TI devices is future roadmap; they have moved away from high performance multicore CPUs and towards multicore ARM SoCs with limited number of c66x cores, which is not an efficient architecture for HPC servers.
Can you recommend a half-size GPU PCIe card ? We need to keep each server at 150 W or less.
Thanks.
-Jeff
Hi Jeff,
I recommend at single slot version of GTX 1050 Ti. They are very efficient in terms of FLOPS/Watts. With the right software, you could also look into the right CPU as GPUs are often not so efficient when it comes to inference, but I think the GTX 1050 Ti would give you enough power to do real-time inference at 75W.
Hi Tim,
Your article about choosing GPU is excellent.
However, I still have the problem. Since my supervisor advise that my future work could focus on deep learning models and I am totally a freshman for this area, I don’t know what problems I will meet and what hardware environment I need. And also, sadly, I didn’t get the budget so I want to know the basic configuration for doing research (dual RTX2070?).
Thanks,
Tony
Two RTX 2070 will probably work well in your case since it is unlikely that you need large amounts of memory if you are still experimenting with deep learning. However, if you want to make sure that you can run all different kind of models you might want to invest in a single RTX 2080 Ti.
Hi ,
I am building a custom deep learning workstation. Your blog suggests buying rtx2070. I would like to know the card that could suit my needs. I am a part-time researcher working on medical image processing. My workloads include dealing with high volume datasets and deriving insights from it through variants of convolutional neural networks.
Both the cards RTX2070 and GTX1080 Ti seems to be better. Which one should i choose?
Your response will be of great help to me
I would recommend either a GTX 1080 Ti or an RTX 2080 Ti since medical imaging dataset often have very large image sizes which consume a lot of memory. It is better to have at least 11 GB of memory. You could also go for a Titan RTX, but the card is not very cost-efficient and I would avoid it if possible.
Hello Tim
I really impressed for your blogs and I want to read more and more. I’m currently master student in data analytics and my final project is about image classification and I tend to use deep learning. I don’t have such fancy computer is laptop suitble to run deep learning? Do you have any suggestion to facilitate this issues ?
You can try the free GPUs from coderlab. This will get you started. Then you can figure out if you want to do more of this and if so, invest in a dedicated desktop with a GPU.
Tim,
Thank you for all the effort in putting your blog together.
I’m building a ML+DL rig for myself to compete in Kaggle etc. I’m set on starting with one RTX2070, and expanding to two based on your posts. I’m debating between going with an Intel (i5) vs AMD (threadripper) architecture for the CPUs. Although CPUs won’t matter for DL, multi-core processing does speed up data prep, and even running cross-validation on xgboost models. I’m primarily an R user but going towards Python as well. What I can’t find is any performance comparison for R/Python libraries on Intel vs AMD.
Any thoughts on this topic?
Cheers!
In general, the Intel software for linear algebra is much more sophisticated than the one from AMD. If you are looking for good XGBoost performance I would recommend an i5 over a Threadripper.
Hi, I have questions about using Bitcoin mining hardware for deep learning. Has someone has researched and written up the tradeoffs? Part of the assumption of the security model for Bitcoin is that miners will not sell cycles to parties intending to damage Bitcoin, as this could cause them to lose their hardware investment if Bitcoin succumbs. However I’ve not seen a good economic discussion suggesting this pans out – if Bitcoin tanks, and the world now has millions of dollars of unused ASICs sitting idle, I would imagine these could be rented out for some heavy DNN training. Is this feasible? If someone was interested in mining and also deep learning, should one buy a mining rig and expect to get value out of it even if mining becomes unprofitable? Also I’m curious if you have any statistics on how much electricity is burned over a period of time (globally) for training.
ASICs are specialized hardware that serves a single purpose. For example, a TPU can compute matrix multiplications. An ASIC for bitcoin mining computes hashes and that is it — it cannot be used for deep learning. If bitcoin tanks it will be useless in any case.
Hi Tim,
great blog and very useful lessons indeed, you can tell you have worked long nights to get this kind of knowledge, good thing that when you enjoy what you do you can spend hours without even noticing!!! To my question: have you done any research regarding egpus and possible bottlenecks due to the thunderbolt 3 interface between the gpu and the laptop cpu? I have seen reviews where they claim due to several issues the mounted gpu will perform at 50% of the nominal capacity for gaming but due to the gpu intensive nature of deep learning I am curious if this still holds or it is not really a problem since most of the processing takes place in the gpu???
Thanks and keep up the good work
eGPUs should be better for deep learning since, unlike in gaming, you do not need to transfer as much data back and forth between the CPU and the GPU. Most of the computation will stay on the GPU. There are some bottlenecks in certain applications like computer vision, but they are less severe than for gaming. I would expect a decrease of 25-30% in performance which is not too bad!
I am planning to set up a deep learning workstation with a colleague. As a motherboard, we want to use a Asus TUF mark I x299, The CPU will be a I9 9820x with 44 PCI lanes.
So, my question is now what GPU would be the best choice for Natural Language Processing like text-to-speech, speech-to-text and other speech recognition models? A Titan RTX is financially impossible, so we thought of the RTX 2080 super (to be launched soon) or the RTX 2080ti, whereas from AMD the Vega VII seems to be quite attractive due to its large VRAM.
Do you have any experience with the hardware concerning compatibility with TF, PyTorch drivers for Linux etc.? Which one would be the best choice? If Nvidia is preferred, is larger VRAM or higher base clock more important for NLP? There is so much contradictory information on the internet that it is hard to discern who really has experience and who just wants to do some marketing (or simply voices prejudice against the companies)
For NLP research I recommend a RTX 2080 Ti. If you want to run large transformers you need that extra memory. 8 GB of RAM can be limiting.
After going through your post on building a deep learning PC , I feel I can revive my old desktop & defer buying an i7 9750H + RTX 2060 based laptop . I am mostly interested in sequence processing like trying out ELMO , BERT , BiLSTM / CRF . Would it be wise to upgrade my i3 3220 by adding an RTX 2070 Super GPU ? The motherboard has 1×16 PCI Express 2.0 slot for GPU support . I have already 2 x 8 GB RAM and 512 GB SATA SSD .
thanks
I like that thinking! I think that could work. A couple of things though: For BERT I would recommend a GPU with more memory (RTX 2080 Ti). Also, you will live with some caveats: The CPU does not have many cores which can become a problem with you need to load large amounts of data with each mini-batch; this can cost 10-20% runtime performance. For NLP this should not be the biggest problem though. Theoretically, PCIe 3.0 GPUs are compatible with PCIe 2.0 motherboards, but once someone commented here that it did not work for him. So theory does not always work out practically. I would recommend keeping an eye on the return policy if you buy a GPU so that you can return it if you find it is not compatible with your motherboard.
Hi , I read your post on deep learning hardware. I am planning to buying to hardware for deep learning research . I am a NLP researcher and also planning venture into Kaggle . I am seeking your advice on RX580 GPU , do you think it is powerful enough for doing NLP + Image modelling. Looking forward for you response
It is powerful but you can get issues with compatibility of libraries. I have no direct experience with AMD cards and there is not much data out there how good the experience is so I cannot give you a full recommendation. If you go ahead with the AMD card it would be great to hear how your experience has been.