Consolidation Of Various Tools

Hi, fed up with keeping track of so many tools of trade ?, Use this article as a quick reference to various tools of common trades (Big Data, IoT, Machine Learning, DEVOPs, jQuery etc.). Here are emphasized high level features, and reference liks for various tools along with reference URLs. So just enjoy good work created with a lot of patiance.

<hr>
**Table of Contents**

[TOCM]

<hr>

<h1>Java SE Development Kit</h1>
Runs same code on multiple platforms (Windows, Unix or MAC) using Java Virtual Machine (a byte-code interpreter). Found at <a href="https://www.oracle.com/index.html" target="_blank">oracle site</a>, in two flavours: JRE or JDK (a.k.a Java SE). It is recommended to install JDK only, both x32 and x64 bit versions on development machines. However, you may use JRE on production systems. Ensure to create Enviorment Variable, JAVA_HOME like C:\Program Files\Java\jdk1.8.0_131. 
Nothing special runs without Java. 
<h1>
Node Version Manager (NVM)</h1>
A package manger specially designed to install, update and change versions of Node.js in Linux systems. Gets installed on windows along with Node.js installation.  It's instructions can be found at <a href="https://github.com/creationix/nvm"><github\nvm></a>.

<h1>Node.js (Asynch I/O) </h1>
Modern web can't be thought of without JavaScript running on the server, as well as in the browser giving a rich multi-platform user experience with rich libraries like bootstrap, masonry, d3.js and 1000's more.
<ul>
<li>Ryan Dahl (Node.js Creator) created this server side library (that can run in docker containers, ubuntu etc.) to create real-time websites with PUSH capability.</li>
<li>It's a tool for working in the non-blocking, event-driven I/O paradigm. Typical use of Node.js is creation of Chat Application
</li>
<li>Node.js® is a JavaScript runtime built around (or is a packaged compilation) of Google’s Chrome V8 JavaScript Engine. The libuv platform abstraction layer, and a core library which itself is primarily written in JavaScript.</li>
<li>In one sentence: <a href="https://www.toptal.com/nodejs/why-the-hell-would-i-use-node-js">Node.js®</a> shines in real-time web applications employing push technology over websockets.</li>
<li>Must see <a href="https://www.toptal.com/nodejs/why-the-hell-would-i-use-node-js" target="_blank">Why The Hell Would I Use Node.js?</a>. Stateless request-response communications had been around for a while but best part about Node.js is real-time, two-way connections, where both the client and server can initiate communication, allowing them to exchange data freely. This is in stark contrast to the typical web response paradigm, where the client always initiates communication. Additionally, it’s all based on the open web stack (HTML, CSS and JS) running over the standard port 80.
</li>
</ul>

<h1>Bower (Web Package Manager)</h1>
<a href="https://bower.io/">Bower</a> is a well known package manager for web. There are 1000's of open-source javascript libraries. Bower tool helps you out in installing, updating, purging, and upgrading the same from command line of your favorite operating system (Ubuntu, Windows, & MAC). Bower reads it's configuration file name bower.json for libraries and versions you wan't to install and deploy them to folder bower_components along with ensuring presence of right dependencies. In fact lets say if you've defined installation of jQuery as "jquery": "^3.2.1", bower will install jQuery v3.2.1 and keeps updating it untill versions 3.*.* without breaking your code as you've specified all versions (^3.2.1).
 
<h1>Grunt & Gulp (JavaScript Task Runner)</h1>
<a href="https://gruntjs.com/">Grunt</a> is a well known task manager for web. There are 1000's of task plugins for grunt that you may like to use in order to save manual effort in delivery of a web product. You have plugin's to Relace text, Move files, Minify & Compress HTML/JS/CSS and lot many things. So cool !!. Grunt process instructions in file: Gruntfile.js releving you of much hassles in manual deliverty process. This is a command line tool for Ubuntu / Windows which can execute multiple tasks defined in Gruntfile.js (executed tasks maintains dependent task sequencing and execution).
<a href="http://gulpjs.com/">Gulp.js or Gulp</a> is another task runner like Grunt with less number of plugins and using Gulpfile.js for task automation requirements. It's good as a second choice as plugins are many.

<h1>WinPython - Scientific Python Distro</h1>
 <a href="https://winpython.github.io/">WinPython</a> is an open-souce distribution for Python Lanaguge for Windows 7/8/10 and Scientific / Educational Usage. It contains following
<ul>
<li>IDELX - Python GUI</li>
<li>IPython Console - IPython Qt Console</li>
<li>Jupyter Notebook</li>
<li>Spyder - Python IDE providing features like MATLAB. It includes debug feature. It's included with Anaconda, WinPython and Python(x,y)</li>
<li>WinPython Command Prompt & Powershell Prompt</li>
<li>Contains NumPy, SciPy, Sympy, Matplotlib, Pandas, Pyqtgraph
</li>
<li>Many installers for x86/x64 and based on Python Versions (2.7, 3.4, 3.6, ...)</li>
<li>Can install multiple version on windows and register into OS using WinPython Control Panel
</li>
<li>ALAS that it won't work after installation on Windows and had average features</li>
</ul>

<h1>Python(x,y)- Scientific Python Distro</h1>
 <a href="https://python-xy.github.io/">Python(x,y)</a> is a free scientific and engineering development software for numerical computations, data analysis and data visualization based on Python programming language, Qt graphical user interfaces and Spyder interactive scientific development environment. Interactive computing with Matplotlib and IPython. GUI design tools (Qt - Eclipse Integration). Scientific developement enviornment with Spyder. No Jupyter NB, but better python packages than WinPython. 
See image for many tools and libraries for Python. Especially for image processing, data processing, etc.
<img src="https://python-xy.github.io/images/pythonxy_2117.png" alt="Python Libraries" width="675" height="380">
 List of included plugins are <a href="https://python-xy.github.io/downloads.html" target="_blank">here</a>.

<h1>Enthought Canopy - Scientific Python Distro</h1>
<a href="https://www.enthought.com/products/canopy/">Enthought Canopy</a> is a comprehensive Python analysis environment that provides easy installation of over 450 core scientific analytic and Python packages, creating a robust platform you can explore, develop, and visualize on.
<ul>
<li>One-Click Python Package Installation with a Graphical Package Manager
</li>
<li>
<a href="http://static.enthought.com/etw/img/python-data-import-tool.jpg">Data Import Tool (NEW !)</a> for importing columnar text files into Pandas DataFrames and creating repeatable data munging scripts
</li>
<li>
Code Editor with Jupyter/IPython Notebook Support
</li>
<li>Interactive <a href="https://static.enthought.com/etw/img/interactive_graphical_python_code_debugger_in_enthought_canopy.png?8feb477">Graphical Python Code Debugger</a> and Variable Browser
</li>
<li>Integrated IPython Prompt
</li>
<li>
Python for Excel with <a href="https://www.enthought.com/products/pyxll/">PyXLL (add-on)</a>: Very Cool -- run python scripts in excel.
</li>
<li>Integration with the Intel MKL and Microsoft Python Tools for Visual Studio</li>
</ul>
See <a href="https://youtu.be/Pd5tw3SDmTM">Financial Portfolio Analysis Video</a> to understand what it can do. <a href="https://www.enthought.com/products/canopy/package-index/">450 packages</a>, Excel Add-in, Jupyter & Debugger

<h1>Anaconda - Data Science Platform</h1>
<a href="https://www.continuum.io/">Anaconda </a>is a Leading Data Science platform powered by Python & includes R (Spark with Clouders/Horton) 
<ul>
<li>Cites to include Jupyter Notebook, R, Sckit-learn, Tensorflow, Theano, Spark, Hadoop, Cloudera, Parquet, JSON.
</li>
<li>
Can be used with <a href="https://www.continuum.io/blog/developer-blog/using-anaconda-h2o-supercharge-machine-learning-predictive-analytics">H2O.ai Libraries</a> (<a href="https://www.continuum.io/blog/developer-blog/using-anaconda-h2o-supercharge-machine-learning-predictive-analytics">Article Here</a>) and <a href="https://www.mapd.com/">MAPD Analytics Platform </a>to supercharge machine learning through GPU based parallel computations. Theano, tensorflow, neon, h2o, keras plus many more</li>
<li>Previously using Microsoft SQL Server was challenge with Python not SQL Server 2017 includes Anaconda packages on server side and connecting is simple - <a href="https://www.continuum.io/blog/developer-blog/anaconda-easy-button-microsoft-sql-server-and-python">see here</a>.</li>
<li>
<a href="https://www.continuum.io/presentations">Presentations</a> on scaling and deep learning, Publically learnable interactive visualizations using Python Library <a href="http://bokeh.pydata.org/en/latest/docs/gallery.html">Bokeh</a> and awsome demonstrations <a href="https://anaconda.org/gallery">apps</a> examples online.
</li>
<li>720+ data science packages for interactive data visualizations, machine learning, deep learning, image processing, NLP and Big Data</li>
<li>World-class package, dependency and environment management, Even deploy to your own cloud offerings. </li>
<li>
Cloudera CDH Parcels that make it a snap to deploy Anaconda to Cloudera Clusters. <a href="https://www.cloudera.com/documentation/enterprise/5-6-x/topics/cdh_intro.html">Cloudera CDH</a> is most complete, tested, and popular distribution of Apache Hadoop and related projects like <a href="https://hortonworks.com/">Hortonworks </a>& <a href="https://mapr.com/">MapR</a>. Cloudera adds Spark to Anaconda.
</li>
<li><a href="https://anaconda.org/">Anaconda comes with own Cloud</a>, Native access to data in HDFS and Amazon S3</li>
<li>Distributed computing of Data Science Workloads using <a href="http://dask.pydata.org/en/latest/">Dask</a>. </li>
<li>Enables Microsoft Excel to leaverage power of Python and R</li>
<li><a href="https://docs.continuum.io/anaconda/ide_integration">Can be integrated with IDEs</a> (Spyder, PyCharm, Visual Studio, Eclipse, IDLE, ...)</li>
</ul>
<h2><a href="https://www.continuum.io/sites/default/files/Anaconda-Business-Subscription-Brochure.pdf">Anaconda Enterprise (Paid)</a></h2>
It contains following products all are paid licensed except #1 (Anaconda)
<ol>
<li>
Anaconda: Data-Science Core (750 + <a href="https://docs.continuum.io/anaconda/pkg-docs">Python Packages</a> & <a href="https://docs.continuum.io/anaconda/r-language-pkg-docs">R Packages</a>), <a href="https://docs.continuum.io/anaconda/navigator/">Anaconda Navigator</a> (Package Manager) with access to R/Python/IDE packages, <a href="https://docs.continuum.io/anaconda/ide_integration">Integrable with IDE</a>, a lot of <a href="https://docs.continuum.io/anaconda/excel">EXCEL Plugins</a>. 
</li>
<li>
<a href="https://docs.continuum.io/anaconda-repository/user/using.html">Anaconda Repository</a>: Anaconda Repository is Continuum Analytics’ Anaconda Cloud for private enterprises. It is package management server software that makes it easy for you to find, access, store and share public and private notebooks, environments, and conda and PyPI packages, and to keep up with updates made to the packages and environments you’re using.</li>
<li>
<a href="https://docs.continuum.io/accelerate/">Anaconda Accelerate</a>: It's a package for several numerical libraries optimized for performance on Intel CPUs and NVidia GPUs. With binding to CUDA Libraries (cuBLAS, cuFFT, cuSPARSE, CUDA Sorting). Speed-boosted linear algebra operations in NumPy, SciPy, scikit-learn and NumExpr libraries using Intel’s Math Kernel Library (<a href="https://docs.continuum.io/accelerate/mkl-overview.html">MKL</a>). Accelerated versions of Numpy's built-in UFuncs and FFT.
</li>
<li>
<a href="https://docs.continuum.io/anaconda-adam/">Anaconda Adam</a>: Anaconda Adam allows users to easily install the Anaconda platform on one or more machines using a command line interface (CLI). It also can install products: Repository, Enterprise Notebooks, & Scale.</li>
<li>
<a href="https://docs.continuum.io/ae-notebooks/">Anaconda Enterprise Notebooks</a>: The AEN platform consists of three main service groups: AEN Server, AEN Gateway, and AEN Compute. These services can be either be distributed across multiple servers (recommended), or run on a single machine.
</li>
<li><a href="https://docs.continuum.io/anaconda/fusion/">Anaconda Fusion</a>: Integrated Anaconda with EXCEL.</li>
<li>
<a href="https://docs.continuum.io/anaconda-scale/">Anaconda Scale</a>:  Anaconda Scale provides functionality to easily deploy Anaconda® packages and a distributed computation framework across a cluster. It helps you manage multiple conda environments and packages–including Python and R–on cluster nodes. Anaconda parcel for Cloudera CDH, including custom-generated parcels. Deployment of conda packages and environments with Spark jobs. 
</li>
<li>
<a href="https://docs.continuum.io/anaconda-cloud/">Anaconda Cloud</a>: Anaconda Cloud is a package management service that makes it easy to find, access, store and share public and private notebooks, environments, and conda and PyPI packages, and to keep up with updates made to the packages and environments you’re using. You can build new packages using the Anaconda Client command line interface (CLI), then manually or automatically upload the packages to Anaconda Cloud to quickly share with others or access yourself from anywhere.
</li>
<li>Sponsored Projects
<ul>
<li>
<a href="https://docs.continuum.io/docs_oss/blaze/">Blaze</a>: The Blaze Ecosystem provides Python users high-level access to efficient computation on inconveniently large data. Translates NumPy/Pandas-like syntax to data computing systems (e.g. database, in-memory, distributed-computing). This provides Python users with a familiar interface to query data living in a variety of other data storage systems. One Blaze query can work across data ranging from a CSV file to a distributed database. Contains: <a class="reference external url-external" href="http://odo.pydata.org/" target="_blank">Odo</a>: Migrates data between formats (CSV, JSON, Databases); <a class="reference external url-external" href="https://github.com/libdynd/libdynd" target="_blank">DyND</a>: In-memory dynamic arrays (variable length strings, ragged arrays, and GPUs.); <a class="reference external url-external" href="http://dask.pydata.org/" target="_blank">Dask.array & Dask.dataframe</a>: Multi-core / on-disk NumPy larger-than-memory arrays/dataframes;  
</li>
<li>
<a href="http://bokeh.pydata.org/en/latest/">Boke</a>:  Bokeh is a Python interactive visualization library that targets modern web browsers for presentation in the style of D3.js, and to extend this capability with high-performance interactivity over very large or streaming datasets.
</li>
<li>
<a href="https://docs.continuum.io/docs_oss/conda/">Conda</a>: Conda is an open source package management system and environment management system for installing multiple versions of software packages and their dependencies and switching easily between them. It works on Linux, OS X and Windows, and was created for Python programs but can package and distribute any software.</li>
<li>
<a href="http://dask.readthedocs.io/en/latest/">Dask</a>: Dask is a flexible parallel computing library for analytic computing. Dynamic task scheduling optimized for computation; “Big Data” collections like parallel arrays, dataframes;</li>
<li>
<a href="https://docs.continuum.io/docs_oss/llvmlite/">LLVMLite</a>: A lightweight LLVM python binding for writing JIT compilers</li>
<li>
<a href="https://phosphorjs.github.io/docs">PhosphorJS</a>: A collection of libraries designed to make it easy to build high-performance, pluggable, desktop-style web applications.</li>
<li>
<a href="http://numba.pydata.org/">Numba</a>: Numba gives you the power to speed up your applications with high performance functions written directly in Python. With a few annotations, array-oriented and math-heavy Python code can be just-in-time compiled to native machine instructions, similar in performance to C, C++ and Fortran, without having to switch languages or Python interpreters.</li>
</ul>
</li>
</ol>