Programme

Outline of the program

Monday, August 26
Tuesday, August 27
  • 9:00 - 15:50 Workshop
Wednesday, August 28
  • 9:00 - 16:20 Workshop
  • 18:00 -  Banquet
    location: Funasei in a Japanese houseboat style
    The bus will take us from the venue to the boatslip in Shinagawa.
Thursday, August 29
  • 9:00 - 11:30 Workshop
► Lunch will be served at Sanjo-Kaigisho on Tuesday and Wednesday.

Programme

Monday, August 26

13:40-13:45 Opening Remark: Kenji Yamanishi

13:45-14:45 Keynote Speach: Jorma Rissanen

15:00-16:20 Session M1 (Information Measure)
Chair: Ralf Eggeling
  • Teemu Roos: Information capacity of full-body movements [abstract]
  • Joachim M. Buhmann: What is the information content of an algorithm? [abstract]
  • Joe Suzuki: The MDL principle for arbitrary data:either discrete or continuous or none of them [abstract][PDF]
16:35-17:30 Session M2 (Data Compression)
Chair: Shigeki Miyake
  • Ionut Schiopu & Ioan Tăbuş: Lossless contour compression using chain-code representations and context tree coding [abstract][PDF]
  • Ioan Tăbuş & Jorma Rissanen: Time- and space-varying context tree models for image coding and analysis [abstract]
18:00- Reception

Tuesday, August 27

9:00-10:00 Plenary Talk: Nicolò Cesa-Bianchi

10:10-11:30 Session Tu1 (Machine Learning)
Chair: Joseph A. O'Sullivan
  • Junya Honda: Regret Analysis of a Bandit Problem for Normal Distributions with Unknown Means and Variances [abstract]
  • Taiji Suzuki: Dual Averaging and Proximal Gradient Descent for Online Alternating Direction Multiplier Method [abstract]
  • Jesus Enrique Garcia & Verónica Andrea González-López : Estimating the structure of interacting coordinates for a multivariate stochastic process [abstract] [PDF]

11:30-12:30 Lunch

12:50-14:10 Session Tu2 (Time Series and Change Detection)
Chair: Narayana Prasad Santhanam
  • Noboru Murata, Kensuke Koshijima & Hideitsu Hino: Distance-based Change-Point Detection with Entropy Estimation [abstract][PDF]
  • Kenji Yamanishi & Hiroki Kanazawa: Stochastic Complexity for Piecewise Stationary Memoryless Sources [abstract]
  • Verónica Andrea González-López: Detecting Regime Changes In Markov Models [PDF]

14:30-15:50 Session Tu3 (Time Series)
Chair: Teemu Roos
  • Narayana Prasad Santhanam: Estimation in Markov processes [abstract]
  • Jun'ichi Takeuchi: Asymptotically Minimax Prediction for Markov Sources [abstract]
  • Eiji Takimoto: Combinatorial online prediction by continuous relaxation [abstract]

Wednesday, August 28

9:00-10:00 Plenary Talk: Toshiyuki Tanaka

10:10-11:30 Session W1 (Channel Coding and Optimization)
Chair: Takafumi Kanamori
  • Sanghee Cho & Andrew Barron: Approximate Iterative Bayes Optimal Estimates for High-Rate Sparse Superposition Codes [abstract][PDF]
  • Cynthia Rush: Using the Method of Nearby Measures in Superposition Coding with a Bernoulli Dictionary [abstract][PDF]
  • Shiro Ikeda: Optimization of probability measure and its applications in information theory [abstract]

11:30-12:30 Lunch

12:50-14:10 Session W2 (Network)
Chair: Junya Honda
  • Ryota Tomioka: Convex Optimization for Tensor Decomposition [abstract][PDF]
  • Wray Buntine: The flow of information in networks of probability vectors [abstract]
  • Shigeki Miyake & Hitoshi Asaeda: Network Coding and Its Application to Content Centric Networking [abstract][PDF]

14:30-16:20 Session W3 (Sequential Prediction)
Chair: Andrew Barron
  • Fumiyasu Komaki: Bayesian predictive densities when the distributions of data and target variables are different [abstract]
  • Kazuho Watanabe, Teemu Roos & Petri Myllymäki: Achievability of Asymptotic Minimax Optimality in Online and Batch Coding [abstract][PDF]
  • Xiao Grace Yang & Andrew Barron: Large Alphabet Coding and Prediction [abstract][PDF]
  • Joseph A. O'Sullivan: Information Bounds and Poisson Inference [abstract]

18:00- Banquet Dinner

Thursday, August 29

9:00-10:00 Plenary Talk: Aapo Hyvärinen

10:10-11:30 Session Th1 ( Latent Variable Models)
Chair: Kazuho Watanabe
  • Ralf Eggeling, Teemu Roos, & Petri Myllymäki : Model Selection In a Setting With Latent Variables [PDF]
  • Keisuke Yamazaki: Distribution-Based Estimation of the Latent Variables and its Accuracy [abstract][PDF]
  • Sumio Watanabe: WAIC and WBIC are Information Criteria for Singular Statistical Model Evaluation [abstract][PDF]

Keynote & Plenary Speakers

Keynote Speaker

Jorma Rissanen
(Helsinki Institute for Information Technology HIIT)

Title: Entropy in Models for Estimation

Abstract: PDF file is here


Plenary Speakers

Nicolò Cesa-Bianchi (The University of Milan)

Title: Bandits without Regrets[PDF]

Abstract: Multi-armed bandit problems formalize the dynamic exploration-exploitation trade-offs arising in sequential decision-making tasks with partial feedback. Online services, such as recommendation systems, are natural targets for bandit algorithms because of the conflicting needs of focusing on users' interests and exploring new options. A bandit algorithm is typically evaluated in terms of regret, measuring how much better the algorithm could have performed if the best option had been chosen at each step. In this talk, I will describe algorithms for regret minimization in various bandit settings, discuss their mathematical properties, and highlight the most important directions of future research.

Aapo Hyvärinen (University of Helsinki)

Title: Estimation of unnormalized statistical models without numerical integration[PDF]

Abstract: Often in a parametric statistical model, the probability density function (pdf) is not properly normalised, i.e. it does not integrate to unity. While a pdf can always be normalised by dividing it by its integral (normalisation constant or partition function), such integration can be computationally extremely difficult in high-dimensional spaces. We have been developing methods for estimation of unnormalized models (in real spaces) which do not require such numerical integration. First, I proposed score matching, in which the computation of the partition function is completely avoided by taking suitable derivatives of the log-pdf. Second, we proposed (with M. Gutmann) noise-contrastive estimation, which starts from the intuitive principle that we train a logistic regressor that learns to discriminate between the data and artificially generated noise; we found that surprisingly, such learning enables estimation of an unnormalised model for the data. Connections between these methods and those by other authors, as well as pros and cons of different methods will be discussed.

Toshiyuki Tanaka (Kyoto University)

Title: Compressed sensing and minimax denoising

Abstract: A sparse vector can be regarded as a collection of samples from a sparse mixture distribution, defined as a mixture of a delta function located at zero and an arbitrary probability distribution. l_1 reconstruction provides an efficient way to reconstruct such a sparse vector from linear measurements on it. Critical compression rate for l_1 reconstruction is defined as the critical value of the ratio of the number of measurements to the dimension of the sparse vector, above which l_1 reconstruction succeeds and below which it fails in the large-system limit, i.e., in the limit where the number of measurements and the dimension are sent to infinity while their ratio is kept finite. In this talk, we describe critical compression rates as a quantity having a role similar to the Shannon entropy. This description is based on our study of the case where one knows prior probabilities for the elements of the sparse vector to take non-zero values. We have studied how to optimize weights in weighted l_1 reconstruction, and have shown that critical compression rates are additive if the weights of l_1 reconstruction are optimally chosen. The relation between critical compression rates and minimax risk of Bayesian denoising in Gaussian channels is also reviewed.