PageRenderTime 38ms CodeModel.GetById 15ms RepoModel.GetById 0ms app.codeStats 0ms

/PROJECT_IDEAS.md

https://gitlab.com/ini-python-lab-course/ss16
Markdown | 126 lines | 68 code | 58 blank | 0 comment | 0 complexity | 0675f45386dbf3faac26fb80e3fc2368 MD5 | raw file
  1. # Brainstorming: libraries
  2. ## Scientific computation
  3. The most important libraries (at least for us) are [NumPy](http://www.numpy.org/) and [SciPy](http://www.scipy.org/). They contain the most important mathematical methods, like:
  4. * linear algebra
  5. * statistics
  6. * signal processing
  7. * optimization
  8. * Fourier Transforms
  9. * ...
  10. There are many others, specialised for specific fields. For instance [Natural Language Toolkit (NLTK)](http://www.nltk.org/) for linguistic tasks.
  11. And then visualization is important, of course. The most popular library is [matplotlib](http://matplotlib.org/).
  12. ## Machine learning
  13. You can find many different ML problems here:
  14. * [Kaggle](https://www.kaggle.com/)
  15. * [UCI Machine Learning Repository](https://archive.ics.uci.edu/ml/datasets.html)
  16. Some useful libraries containing different ML methods:
  17. * [scikit-learn](http://scikit-learn.org/stable/)
  18. * [Modular Toolkit for Data Processing (MDP)](http://mdp-toolkit.sourceforge.net/): It's not very actively developed anymore but we still use it a lot in our workgroup.
  19. ## (Social) network analysis
  20. There are several Python libraries for (social) network analysis like [NetworkX](http://networkx.github.io/) or [graph-tool](https://graph-tool.skewed.de/). Datasets containing networks can found for instance here:
  21. * [Stanford Large Network Dataset Collection](http://snap.stanford.edu/data/index.html)
  22. * [Gephi Wiki](https://github.com/gephi/gephi/wiki/Datasets)
  23. * [Social Graphs in Movies](http://moviegalaxies.com/)
  24. ## IT-Security
  25. There are crypto libraries like [cryptography](https://cryptography.io/en/latest/) and [PyCrypto](https://www.dlitz.net/software/pycrypto/).
  26. ## Scraping websites
  27. * [import.io](https://import.io/): Service that extracts data from websites
  28. * [BeautifulSoup](http://www.crummy.com/software/BeautifulSoup/): Convenient access to content of a downloaded website
  29. * [Scrapy](http://scrapy.org/): Framework for scraping websites
  30. * [Selenium](http://www.seleniumhq.org/): Allows complete automation of a browser via script
  31. Think of data sources like concert tickets or products, movies (IMDB)...
  32. Note however, that many websites don't need to be scraped because they offer a proper API to access their content. Here are examples from a [long list](http://www.programmableweb.com/category/all/apis?order=field_popularity) with some of the most popular web APIs:
  33. * Google Maps, Twitter, YouTube, Flickr, Facebook, Amazon Product Advertising, Twillo, Last.fm, eBay, ...
  34. ## Web development
  35. There are Python frameworks for developing websites, i.e., organizing all the server-side logic and databases with Python. Two notable framworks are [Django](https://www.djangoproject.com/) (complex) and [WebPy](http://webpy.org/) (light-weight).
  36. # Examples
  37. ## Virtual rat hippocampus
  38. ![Place cell architecture](http://www.scholarpedia.org/w/images/thumb/9/94/HierarchicalSFAModel.jpg/400px-HierarchicalSFAModel.jpg)
  39. ![Plce cell result](http://www.scholarpedia.org/w/images/thumb/9/96/ModelCells.jpg/200px-ModelCells.jpg)
  40. That's something we are doing a lot in our workgroup. If you like to try something like this, ask Fabian!
  41. ## Google's dreaming neural networks
  42. Google realeased the Python scripts for it's famous dreaming neural networks ([deepdream](https://github.com/google/deepdream)). Others have build on that, for instance [making *Fear and Loathing in Las Vegas* even more uncanny](https://github.com/graphific/DeepDreamVideo):
  43. ![Fear and Loathing example](https://camo.githubusercontent.com/dcf15823a576975a5bd2d1af1696a25a07b7e6aa/687474703a2f2f6d656469612e67697068792e636f6d2f6d656469612f6c34316c537a6a5473474a63497a704b672f67697068792e676966)
  44. Here's another project that created a simple user interface: [bat-country](https://github.com/jrosebr1/bat-country)
  45. ## WhatsApp chat bots
  46. Someone wrote [yowsup](https://github.com/tgalal/yowsup), an Python wrapper for the (unoffcial) WhatsApp API. People used this for instance to write chat bots ([yowlayer-cleverbot](https://github.com/tgalal/yowlayer-cleverbot), [WAbot](https://github.com/KaveenR/WAbot)) for WhatsApp.
  47. ## Automating Tinder with Eigenfaces
  48. Someone wrote a (Java) script to automatically find matches for him/her using Eigenfaces: [tinderbox](https://github.com/crockpotveggies/tinderbox)
  49. ![two eigenfaces](https://raw.githubusercontent.com/crockpotveggies/tinderbox/master/public/img/tinderbox_eigenfaces_models.jpg)
  50. ## Cheap ticket notification
  51. A nice example for scraping websites that don't have an API: [LTUR-Notifer](https://github.com/policecar/ltur-notifier) scrapes information from [bahn.ltur.com](bahn.ltur.com) and sends email or push notification to a smartphone when a new cheap ticket appears.
  52. ## Packet sniffing and injection
  53. Also people seem to use Python scripts for [packet sniffing and injection](http://suryamattu.com/PACKET-SNIFFING-TUTORIAL), for instance.
  54. ## Implementing Machine learning algorithms on your own
  55. Basically all you need is numpy and the corresponding math.
  56. Examples: Feed Forward Neural Networks, Auto encoders, Sparse Coding, Baysian Networks, Restricted Boltzmann machines, ...
  57. ## Solving a machine learning problem
  58. Using existing libs like scikit the focus would be on solving a particular task in a good way.
  59. So it would be problem driven and you should have a concrete idea about the problem
  60. Have a look at http://scikit-learn.org/stable/index.html for examples for clustering, resgression, classification, ...
  61. Examples: Object detection (Eye, hand, ...), Spam detection, ...
  62. ## Raspberry Pi
  63. Another place to turn your Python skills into fun and usefull projects it the [Raspberry Pi](https://www.raspberrypi.org/). Some random projects include
  64. * [High-altitude photography](http://opensource.com/life/15/9/pi-sky-high-altitude-ballooning-raspberry-pi),
  65. * [Crypto currency trackers](http://makezine.com/projects/make-43/crypto-currency-tracker/),
  66. * [Dirty dish detectors](http://makezine.com/projects/make-43/dirty-dish-detector/), and
  67. * [Pirate Radios](http://makezine.com/projects/raspberry-pirate-radio/).
  68. You can find many more projects for inspiration on [Make](http://makezine.com/?s=raspberry) and [Instructables](http://www.instructables.com/id/Raspberry-Pi-Projects/).