Phobrain

Image Analysis 4 - Siamese network comparison of histograms

Starting point (now gone, maybe will be replaced?): Train a Siamese MLP on pairs of digits from the MNIST dataset.

Using various image histogram methods applied to ~30,000 selected pairs of images to train a Siamese neural net, 5x Dense Relu with dropout, width adjustable, using Keras with Tensorflow, from 16K photos, including photos from Bill, Ellen and Raf & Skot.

Results are visible by going to the View page, then Phob->Search Mode, and phi1/phi2 icons appear with other options. Phi1 is convolutional 48x48 Hue*Saturation, numbers not shown yet, Phi2 is based on Dense*5 pinch3, below.

Pairs are split into vertical, horizontal groups. Training all pairs and each group separately, on 50:50 positive:negative cases. Testing on 20:80 ratio seen in practice. Cutoff adjusted from original from 0.5 to 0.1:

        x = labels[predictions.ravel() < 0.1].

It seems that the predictions are measures of closeness (0==identical) rather than matches to the 0/1 labels used for indicating not_interesting/interesting, as I originally assumed.

Vertical: train.pos: 14400 train.neg: 14400 test.pos: 1599 test.neg: 7995
Horizontal: train.pos: 13888 train.neg: 13888 test.pos: 1542 test.neg: 7710
Both: train.pos: 28288 train.neg: 28288 test.pos: 3141 test.neg: 15705
Timing: 7 threads used to load files.

Models
(they can't help it if they are beautiful)

Dense*5
Dense*5 pinch
Dense*5 pinch2
Dense*5 pinch3
Dense*5 pinch4

System characteristics

Notes GreyscaleHue*Sat 24RGB 12 Hue*Sat 48RGB 24RGB 32
Data set size 70M177M231M 522M1.3G2.9G
Input width 1505761728 23041382432768
Dense width 128256728 81220484096
Time to load data + train on 2015 laptop 2m3m15m 22m--
Time to load data + train on 1080 ti 1m1m1m 2m9m25-50m
Time to process 78.9M pairs with gpu (7-thread data read) --- 40m (41s)-7h (14m)


Test set accuracy (% vert, horiz, both) for various params

Training accuracy 78-100%, usually 98+.

ModelNotes GreyscaleHue*Sat 24RGB 12 Hue*Sat 48RGB 24RGB 32
Dense*5batch 128, epochs 50 (vert, horiz) 42, 3052, 4048, 48 46, 4643, 5649, 43
Dense*5batch 1024, epochs 100 38, 3152, 3845, 57 65, 5650, 5568, 47
Dense*5/pinchbatch 128, epochs 50 33, 32, 40-- 40, 48, 52-49, 42, 64
Dense*5/pinchbatch 1024, epochs 100 --- 75, 55, 64-48, 58, 60
Dense*5/pinch2batch 1024, epochs 100 --- 63, 55, 64-61, 51, 51
Dense*5/pinch3; rgb32 is phi2 on sitebatch 1024, epochs 100 --- 52-82, 61, 70-76, 61, 60
Dense*5/pinch31024, 100; train both, test v/h (overlap) --- 86-91, 83--
Dense*5/pinch31024, 100; train both, test v/h (no overlap) --- 44-55, 40--
Dense*5/pinch4batch 1024, epochs 100 --- 47-70, 55, 61-68--


Distributions of the raw predictions, scaled to integers.

HS48 vertical pairs



HS48 horizontal pairs



RGB32 vertical pairs



RGB32 horizontal pairs






Software
Histograms from BoofCV.
Neural network: Keras with Tensorflow.



What use to cry for Capricorn? it sails
Across the heart's red atlas: it is found
Only within the ribs, where all the tails
The tempest has are whisking it around.


— Mervyn Peake, Titus Alone

© 2015,2016,2017 Photoriot.