Data & AI Digest for 2018-09-26

In my recent conversations with customers, they have shared the security challenges they are facing on-premises. These challenges include recruiting and retaining security experts, quickly responding to an increasing number of threats, and ensuring that their security policies are meeting their compliance requirements.
More details at…

We are excited to preview a set of Azure Resource Manager Application Program Interfaces (ARM APIs) to view cost and usage information in the context of a management group for Enterprise Customers. Azure customers can utilize management groups today to place subscriptions into containers for organization within a defined business hierarchy.
More details at…

This week at Microsoft Ignite 2018, we are excited to announce seven new features in Azure Stream Analytics (ASA).
More details at…

Today, we are excited to announce the preview of the new Azure HDInsight Management SDK. This preview SDK brings support for new languages and can be used to easily manage your HDInsight clusters.
More details at…

Today I am excited to announce the private preview of Azure VM Image Builder, a service which allows users to have an image building pipeline in Azure. Creating standardized virtual machine (VM) images allow organizations to migrate to the cloud and ensure consistency in the deployments. Users commonly want VMs to include predefined security and configuration settings as well as application software they own.
More details at…

We are happy to share the release of additions and enhancements to Bing Custom Search. Bing Custom Search is an easy-to-use, ad-free search solution that enables users to build a search experience and query content on their specific site, or across a hand-picked set of websites or domains.
More details at…

Today we are extending Try Cosmos DB for 30 days free! Try Cosmos DB allows anyone to play with Azure Cosmos DB, with no Azure sign-up required and at no charge, for 30 days with ability to renew unlimited times.
More details at…

SQL Information Protection policy can now be managed in Azure Security Center. The policy management enables centrally defining customized classification and labeling policies, that will be applied across all databases on your tenant.
More details at…

Azure Stack features a growing independent software vendor (ISV) community that operates across a broad spectrum of environments, empowering you to create compelling and powerful solutions.
More details at…

Earlier this year, we released the public preview of Azure Serial Console for Virtual Machines (VMs). Today we are announcing the general availability of Serial Console and we’ve added numerous features and enhanced performance to make Serial Console an even better tool for systems administrators, IT administrators, DevOps engineers, VM administrators, and systems engineers.
More details at…

Recently I began to look further into Time Series(TS). During the course of my Master’s degree, I used the forecast package quite a bit (Thanks to Prof. Hyndman), and TS got my attention. So, after reading lots of publications about everything you can imagine about TS, I came across one publication from Prof. Eamonn, of the University of … Continue reading Time Series with Matrix Profile
More details at…

Here is the course link.
Course Description
Linear regression serves as a workhorse of statistics, but cannot handle some types of complex data. A generalized linear model (GLM) expands upon linear regression to include non-normal distributions inc…
More details at…

Package developers relaxed a bit in August.; only 160 new packages went to CRAN that month. Here are my “Top 40” picks organized into seven categories: Data, Machine Learning, Science, Statistics, Time Series, Utilities, and Visualization.


nsapi v0.1.1: Provides an interface to the Nederlandse Spoorwegen (Dutch Railways) API, allowing users to download current departure times, disruptions and engineering work, the station list, and travel recommendations from station to station. There is a vignette.

repec v0.1.0: Provides utilities for accessing RePEc (Research Papers in Economics) through a RESTful API. You can request an access code and get detailed information here.

rfacebookstat v1.8.3: Implements an interface to the Facebook Marketing API, allowing users to load data by campaigns, ads, ad sets, and insights.

UCSCXenaTools v0.2.4: Provides access to data sets from UCSC Xena data hubs, which are a collection of UCSC-hosted public databases.

ZipRadius v1.0.1: Generates a data frame of US zip codes and their distance to the given zip code, when given a starting zip code and a radius in miles. Also includes functions for use with choroplethrZip, which are detailed in the vignette.

Machine Learning

dials v0.0.1: Provides tools for creating model parameters that cannot be directly estimated from the data. There is a vignette.

tosca v0.1-2: Provides a framework for statistical analysis in content analysis. See the vignette for details.

tsmap v0.3.1: Implements the Matrix Profile concept for classification.


DSAIRM v0.4.0: Provides a collection of Shiny apps that implement dynamical systems simulations to explore within-host immune response scenarios. See the package Tutorial.

epiflows v0.2.0: Provides functions and classes designed to handle and visualize epidemiological flows between locations, as well as a statistical method for predicting disease spread from flow data initially described in Dorigatti et al. (2017). For more information, see the RECON toolkit for outbreak analysis. There is an Overview and a vignette on Data Preparation.

fieldRS v0.1.1: Provides functions for remote-sensing field work using best practices suggested by Olofsson et al. (2014). See the vignette for details.

Rnmr1D v1.2.1: Provides functions to perform the complete processing of proton nuclear magnetic resonance spectra from the free induction decay raw data. For details see Jacob et al. (2017) and the vignette.


bcaboot v0.2-1: Provides functions to compute bootstrap confidence intervals in an almost automatic fashion. See the vignette.

bivariate v0.2.2: Contains convenience functions for constructing and plotting bivariate probability distributions. See the vignette for details.

DesignLibrary v0.1.1: Provides a simple interface to build designs and allow users to compare performance of a given design across a range of combinations of parameters, such as effect size, sample size, and assignment probabilities. Look here for more information.

doremi v0.1.0: Provides functions to fit the dynamics of a regulated system experiencing exogenous inputs using differential equations and linear mixed-effects regressions to estimate the characteristic parameters of the equation. See the vignette.

eikosograms v0.1.1: An eikosogram (probability picture from the ancient Greek εὶκὀσ – likely or probable) divides the unit square into rectangular regions whose areas, sides, and widths represent various probabilities associated with the values of one or more categorical variates. For a discussion on the eikosogram and its superiority to Venn diagrams in teaching probability, see Cherry and Oldford (2003), and for a discussion of its value in exploring conditional independence structure and relation to graphical and log-linear models, see Oldford (2003). There is an Introduction and vignettes on Data Analysis and Independence Relations.

localIV v0.1.0: Provides functions to estimate marginal treatment effects using local instrumental variables. See Heckman et al. (2006) and Zhou and Xie (2018) for background.

merlin v0.0.1: Provides functions to fit linear, non-linear, and user-defined mixed effects regression models following the framework developed by Crowther (2017). See the vignette for details.

MRFcov v1.0.35: Provides functions to approximate node interaction parameters of Markov Random Fields graphical networks. The general methods are described in Clark et al. (2018). There are vignettes on Preparing Datasets, Gaussian and Poisson Fields, and an example using Bird parasite data.

SCPME v1.0: Provides functions to estimate a penalized precision matrix via an augmented ADMM algorithm as described in Molstad and Rothman (2018). There is a Tutorial and a vignette describing Algorithm Details.

survxai v0.2.0: Contains functions for creating a unified representation of survival models, which can be further processed by various survival explainers. There are vignettes on Local explanations, global explanations, comparing models, and on a custom prediction function.

Time Series

hpiR v0.2.0: Provides functions to compute house price indexes and series, and evaluate index goodness based on accuracy, volatility and revision statistics. For the background on model construction, see Case and Quigley (1991), and for hedonic pricing models, see Bourassa et al. (2006). There is an an introduction to the package and a vignette on Classes.

STMotif v0.1.1: Provides functions to identify motifs (previously identified sub-sequences) in spatial-time series. There are vignettes on motif discovery, examples, candidate generation, and candidate validation.

trawl v0.2.1: Contains functions for simulating and estimating integer-valued trawl processes as described in Veraart (2018), and for simulating random vectors from the bivariate negative binomial and the bi- and trivariate logarithmic series distributions. There is a vignette on trawl processes, and another on the binomial distributions.


arkdb v0.0.3: Provides functions for exporting tables from relational database connections into compressed text files, and streaming those text files back into a database without requiring the whole table to fit in working memory. See the vignette for a tutorial.

aws.kms v0.1.2: Implements an interface to AWS Key Management Service, a cloud service for managing encryption keys. See the README for details.

DatapackageR v0.15.3: Provides a framework to help construct R data packages in a reproducible manner. It maintains data provenance by turning the data-processing scripts into package vignettes, as well as enforcing documentation and version checking of included data objects. There is a Guide to using the package, and a vignette on YAML configuration.

hedgehog v0.1: Enables users to test properties of their programs against randomly generated input, providing far superior test coverage compared to unit testing. There is a general tutorial and a description of the Hedgehog state machine.

jsonstat v0.0.2: Implements an interface to JSON-stat, a simple, lightweight ‘JSON’ format for data dissemination. There is a short quickstart quide.

nseval v0.4: Provides an API for Lazy and Non-Standard Evaluation with facilities to capture, inspect, manipulate, and create lazy values (promises), “…” lists, and active calls. See README.

runner v0.1.0: Provides running functions (windowed, rolling, cumulative) with varying window size and missing handling options for R vectors. See the vignette for details.

RTest v1.1.9.0: Provides an XML-based testing framework for automated component tests of R packages developed for a regulatory environment. There is a short vignette.

sparkbq v0.1.0: Extends sparklyr by providing integration with Google BigQuery. It supports direct import/export from/to BigQuery, as well as intermediate data extraction from Google Cloud Storage. See README.

vapour v0.1.0: Provides low-level access to GDAL, the Geospatial Data Abstraction Library. There is a vignette.


mapdeck v0.1.0: Provides a mechanism to plot interactive maps using Mapbox GL, a JavaScript library for interactive maps, and, a JavaScript library which uses WebGL for visualizing large data sets. The vignette explains how to use the package.

rayshader v0.5.1: Provides functions that use a combination of raytracing, spherical texture mapping, lambertian reflectance, and ambient occlusion to produce hillshades of elevation matrices. Includes water-detection and layering functions, programmable color palette generation, built-in textures, 2D and 3D plotting options, and more. See README for details and examples.

sigmajs v0.1.1: Provides an interface to the sigma.js graph-visualization library, including animations, plugins, and shiny proxies. There is a brief Get Started Guide, and vignettes on Animation, Buttons, Coloring by Cluster, Dynamic graphs, igraph & gexf, Layout, Plugins, Settings, Shiny, and Crosstalk.

survsup v0.0.1: Implements functions to plot survival curves. The vignette provides examples.

tidybayes v1.0.1: Provides functions for composing data and extracting, manipulating, and visualizing posterior draws from Bayesian models (JAGS, Stan, rstanarm, brms, MCMCglmm, coda, …) in a tidy data format. There is a vignette on Using tidy data with Bayesian Models, and vignettes for brms and rstanarm models.
More details at…

This entry was posted in Data & AI Digest and tagged . Bookmark the permalink.