The Sixth Workshop on
Information Theoretic Methods
in Science and Engineering

26-29 August 2013, Tokyo, Japan

# Proceedings

## Contents

Preface |  | i

Jorma Rissanen, Teemu Roos & Petri Myllymäki
Entropy in Models for Estimation |  | 1

Teemu Roos
Information capacity of full-body movements |  | 2

Joachim M. Buhmann
What is the information content of an algorithm? |  | 3

Joe Suzuki
The MDL principle for arbitrary data:either discrete or continuous or none of them |  | 4

Ionut Schiopu & Ioan Tăbuş
Lossless contour compression using chain-code representations and context tree coding |  | 6

Ioan Tăbuş & Jorma Rissanen
Time- and space-varying context tree models for image coding and analysis |  | 14

Nicolò Cesa-Bianchi
Bandits without Regrets: The Power of Adaptive Adversaries |  | 15

Junya Honda
Regret Analysis of a Bandit Problem for Normal Distributions with Unknown Means and Variances |  | 16

Taiji Suzuki
Dual Averaging and Proximal Gradient Descent for Online Alternating Direction Multiplier Method |  | 17

Jesus Enrique Garcia & Verónica Andrea González-López
Estimating the structure of interacting coordinates for a multivariate stochastic process |  | 18

Noboru Murata, Kensuke Koshijima & Hideitsu Hino
Distance-based Change-Point Detection with Entropy Estimation |  | 22

Kenji Yamanishi & Hiroki Kanazawa
Stochastic Complexity for Piecewise Stationary Memoryless Sources |  | 26

Jesus Enrique Garcia & Verónica Andrea González-López
Detecting Regime Changes In Markov Models |  | 27

Narayana Prasad Santhanam
Estimation in Markov processes |  | 31

Jun'ichi Takeuchi
Asymptotically Minimax Prediction for Markov Sources |  | 32

Eiji Takimoto
Combinatorial online prediction by continuous relaxation |  | 33

Toshiyuki Tanaka
Compressed sensing and minimax denoising |  | 34

Sanghee Cho & Andrew Barron
Approximate Iterative Bayes Optimal Estimates for High-Rate Sparse Superposition Codes |  | 35

Cynthia Rush & Andrew Barron
Using the Method of Nearby Measures in Superposition Coding with a Bernoulli Dictionary |  | 43

Shiro Ikeda
Optimization of probability measure and its applications in information theory |  | 49

Ryota Tomioka
Convex Optimization for Tensor Decomposition |  | 50

Wray Buntine
The flow of information in networks of probability vectors |  | 54

Shigeki Miyake & Hitoshi Asaeda
Network Coding and Its Application to Content Centric Networking |  | 55

Fumiyasu Komaki
Bayesian predictive densities when the distributions of data and target variables are different |  | 62

Kazuho Watanabe, Teemu Roos & Petri Myllymäki
Achievability of Asymptotic Minimax Optimality in Online and Batch Coding |  | 63

Xiao Grace Yang & Andrew Barron
Large Alphabet Coding And Prediction Through Poissonization And Titling |  | 68

Joseph A. O'Sullivan
Information Bounds and Poisson Inference |  | 75

Michael U. Gutmann & Aapo Hyvärinen
Estimation of unnormalized statistical models without numerical integration |  | 76

Ralf Eggeling, Teemu Roos, Petri Myllymäki & Ivo Grosse
Model Selection In a Setting With Latent Variables |  | 84

Keisuke Yamazaki
Distribution-Based Estimation of theLatent Variables and its Accuracy |  | 87

Sumio Watanabe
WAIC and WBIC are Information Criteria for Singular Statistical Model Evaluation |  | 90