cannot reproduce the simulated image by using the simulator in cisTEM of alpha version

[chhuangAli]

Dear all,
I was using the simulator in cisTEM of alpha version with the tutorial written on the cisTEM-dev website: https://bhimes.github.io/cisTEM_docs/docs/sim/tutorials/tutorials.html .
In that tutorial, section 3, calculate a stack of noisy particles shows steps to simulate an isolated protein. I followed the instructions and I could reproduce Figures 1 A and B by modifying and executing bgal_flat.sh on my ubuntu 20.04. However, for Figure 1 C, after removing the flag, --save-detector-wavefunction, from optional_args in shell script (bgal_flat.sh) and executing the shell script, the generated image does not look identical with Figure 1 C which is very dark. In fact the generated image looks similar to Figure 1 B. I have no clue why this difference happened. Could you help me with this situation ?

Best,
Chun Lu

Below is the additional information on the issue that I encountered.

To Reproduce
Steps to reproduce the behavior:
0. Download alpha version of cisTEM from https://bhimes.github.io/cisTEM_docs/docs/sim/tutorials/get_cistem.html to a folder, let's say cisTEM_alpha/

1. Move to cisTEM_alpha/ and create a PDB folder to download a PDB and a run script files with "wget https://github.com/bHimes/cisTEM_docs/raw/main/tutorial_materials/bgal_flat.pdb https://github.com/bHimes/cisTEM_docs/raw/main/tutorial_materials/bgal_flat.sh"
2. Move to PDB folder and modify the variables and flags, path_to_cistem="path_to_cisTEM_alpha/src" output_filename="betgal_1elec_per_angSq.mrc", water_scaling=1.0, and optional_args=" --skip-random-angles " in bgal_flat.sh
3. Launch by typing ./bgal_flat.sh
4. The generated image with the name, betgal_1elec_per_angSq.mrc, is generated.

Expected behavior
That generated image is supposed to look like Figure 1C shown on https://bhimes.github.io/cisTEM_docs/docs/sim/tutorials/tutorials.html
But my resulted image does not but looks like Figure 1B.

Screenshots
My resulted image can be found through this link: https://cistem.org/sites/default/files/Simulated_betagal_in_water_with_low_dose.png

Desktop (please complete the following information):

• OS: ubuntu 20.04

Additional context
My output message on terminal after running bgal_flat.sh is written below:
** Welcome to Simulator **

         Version : 0.25
        Compiled : May  5 2021
            Mode : Scripted

Output filename : betgal_1elec_per_angSq.mrc
Make a 3d scattering potential? : -320
Number of threads : 16
PDB file name : bgal_flat.pdb
Add another type of particle? : no
Output pixel size (Angstroms) : 1.0
Linear scaling of per atom bFactor : 1.0
Per atom (xtal) bFactor added to all atoms : 4.0

            wanted size is 320

Create a tilt-series as well? : no
Create a particle stack? : 1
Making a particle stack with 1 images
wanted defocus (Angstroms) : 8000
wanted additional phase shift x * PI radians, i.e. 0.5 for PI/2 shift. : 0electrons/Ang^2 in a frame at the specimen : 1.5
electrons/Pixel/sec : 3.0
number of frames per movie (micrograph or tilt) : 1
Set expert options? : yes
Use an existing set of orientations : no
Linear scaling water intensity : 1.0
fraction of the defocus to add as astigmatism : 0.0
Accelrating volatage (kV) : 300
Objective aperture diameter in micron : 100
Spherical aberration constant in millimeters : 2.7
Std deviation of error to use in shifts, astigmatism, rotations etc. : 0.0Pre-exposure in electron/A^2 : 0
minimum padding of images with solvent : 32
minimum thickness in Z : 10
Propagation distance in angstrom : 5
Rotation of tilt-axis from Y (degrees) : 0.0
Standard deviation on angles in plane (degrees) : 2.0
Standard deviation on tilt-angles (degrees) : 0.1
Standard deviation on magnification (fraction) : 0.0001
Beam-tilt in X (milli radian) : 0.0
Beam-tilt in Y (milli radian) : 0.0
Beam-tilt particle shift in X (Angstrom) : 0.0
Beam-tilt particle shift in Y (Angstrom) : 0.0
Max particles is 0 here

In constructor real total current 32824 32824 32824

Number of particles initialized 1

PDB center of mass at 93.002000 93.002000 93.002000 (x,y,z Angstrom)

Setting origin there.

There are 32824 non-water atoms in the specimen.

Current number of PDBs 1
Using extra phase shift of 0.000000 radians
Got here emulate is 0.000000

There are 1 tilts
for Tilt 0, scaling the pixel size from 1.000 to 1.000

Working on iParticle 0/ 1

For the particle stack, stretching the defocus by 0.00 percent and randmozing the astigmatism angle -90,90Scaling the defocus by 1.000000 to match the def at 300 KeV

AVG BFACTOR FROM PDB IS 19.164429

max/min xyz, 90.431000,-90.431000 68.182007,-68.181999 45.899002,-45.898998
min is 10.000000, shift is 0, max_depth is 91

    for frame 0 the size neigborhood is 2

using neighborhood of 125.00 vox^3 for waters and 125.00 vox^3 for non-waters
Starting projected water calc with sizeN 5, 5
Finishing projected water calc

            wanted size is 407

Is single particle is 1
size pre rot padding 380 335 0.000000 rot
size post rot 1 padding 787 742 0.000000 rot
size post rot 2 padding 787 742 padX 407 padY 407 padZ 0 rot
vol dimension in Ang 787.00 x 742.00 y 104.00 z

    Drift for iTilt 0, iFrame 0 is 0.0000 Ang

Atoms per nm^3 31.422, vol (in Ang^3) 787.00 742.00 104.00
specimen volume is 6.073e+04 nm expecting 1.908e+06 waters
cuttoff is 1.908314e+06 5.925180e+07 0.967793 -0.967793
waters added 1.933e+06 (101.28%)
Using a rmsd of 2.250000 for water perturbation
wZ 104 csZ 104,rotZ 104

Working on iTilt 0 at 119.805107 degrees for frame 0
rotated_Z 104 nSlabs 21

nSlabs 21
iSlab 6
lSlab 5

Found a scattering cetner of mass at -53.511 Ang

Exposure is 1.500 for frame
-53.511456 -5.000000 -2.476191
Propagator distance is -5.000e+00 Angstroms, with offset for CTF of -5.101e+01 Angstroms for the specimen.

    0 out of bounds of 32824 = percent

8.00000e+03 8.00000e+03

Astigmatism params are 0.000e+00 8.365e+01

SEED is 1639468828480218300e

./simulator_ctf_303DESKTOP-2HAG7BR.sh: line 3: ctffind: command not found
tail: cannot open 'fit_303DESKTOP-2HAG7BR_1.txt.txt' for reading: No such file or directory
Failed on the following record :
From Line :
From /groups/grigorieff/home/himesb/cisTEM_downstream_bah/src/core/numeric_text_file.cpp:261
void NumericTextFile::ReadLine(float *)

[chhuangAli]

Hi,

In order to overcome the simulation issue that I described above, I tried two ways:

1. Because I was using the binary for the simulator alone in a folder, the simulator couldn't find the ctffind. I then put ctffind file together with simulator, and launched the simulation again. But the simulator didn't find the ctffind still. Actually, it needs a command, /absolute-path/cttfind, instead of ctffind. This means I need to modify the source code, simulate.cpp , to put 'absolute-path' , which is my second way described below.
2. In simulate.cpp, I created a new flag, named as "ctffind-location", by using a wxString variable specifying the absolute path to the ctffind. The variable is passed to WaveFunctionPropagator::ReturnImageContrast() where ctffind is called.

After implementing the second modification in the codes, the simulator can call ctffind to determine defocus parameters at first stage during the simulation. Also, the simulation can finsh properly.

[bHimes]

Thanks for troubleshooting this @chhuangAli. I'll leave the issue open so that I can amend the tutorial docs/scripts to prevent this in the future.

I am curious though - why go to all the effort of modifying the source code rather than just modifying your environmental PATH variable?

Once you added the ctffind binary to the same folder as your simulate binary, it would just be a simple

export PATH=/path/to/alpha:$PATH

in the script you use to run the simulation.

[chhuangAli]

Thanks for the reply @bHimes.
I included the simulator path into the envrionment variable, PATH and executed the binary of simulator, which returned the reasonable simulated images.
I was thinking to add a path variable is useful because there is a shell script written all of the input variables into the simulation. So the user can also set the path in the shell script without specifying the simulator path whenever execution. Also, I was interested in the working mechanism inside the simulator in the aspect of techinique and physics, for example, the wanted size of simulation (I was confused the negative size at begining when I wanted to increase larger size of the simulated movie frame. ) so I modified directly the source code.