training prototxt file reading only limited entries in lmdb file

[adhara123007]

Dear Sir,

I installed your software. I have 50000 entries in my lmdb file but when I run the training prototxt file, it reads only 13000 files.
Can you please let me know the reason for that? What checks should I perform?

Thanks and Regards,
Arnab

[nikolausmayer]

We never used the lmdb format — I'm afraid I can't help here.

[adhara123007]

Hi Nikolaus,

But your training protxt file says lmdb. This is from the FlowNet2-CSS_train.prototxt

layer {
name: "CustomData1"
type: "CustomData"
top: "blob0" # Image 0
top: "blob1" # Image 1
top: "blob2" # Flow ground truth
top: "unused1" # Occlusions, remove if not present
include {
phase: TRAIN
}
data_param {
# THIS IS ONLY AN EXAMPLE:
# replace the following lines with the dataset you really want to use:
source: "../../data/FlyingThings3D_release_TRAIN_lmdb"
batch_size: 8
backend: LMDB
preselection_label: 1
rand_permute: true
rand_permute_seed: 77
slice_point: 3
slice_point: 6
slice_point: 8
encoding: UINT8
encoding: UINT8
encoding: UINT16FLOW
encoding: BOOL1
verbose: true
}
}

[nikolausmayer]

Yes, but I don't actually now what the "CustomData" layer is and who wrote it 😕. I never used it, and we always use our own custom storage solution.
Have you tried the official tutorial which uses LMDB on MNIST?

[xianshunw]

@adhara123007 how is your problem solved?