Demonstrate idea in action by making two scripts that will then make three other script files each with random names and content and run each of the eight total scripts only once.
Sessions where this notebook was developed , and thus should be able to be run in similarly spawned sessions, can be obtained by going here and pressing launch binder.
Need shortuuid ackage to make making the random scripts and content easily.
%pip install shortuuid
Requirement already satisfied: shortuuid in /srv/conda/envs/notebook/lib/python3.7/site-packages (1.0.11) Note: you may need to restart the kernel to use updated packages.
s='''script_name = 'make_three_scripts.py'
import shortuuid
for x in range(1,4):
output_file_name_prefix = shortuuid.ShortUUID().random(length=10)
output_file_name = f"{output_file_name_prefix}.py"
so = f"print('I was output by script number {x} of 3 made by {script_name};"
so += f" generated by {output_file_name}')"
with open(output_file_name, 'w') as output_file:
output_file.write(so)'''
%store s >make_three_scripts.py
!sed 's/make_three_scripts.py/make_three_MORE_scripts.py/g' <make_three_scripts.py >make_three_MORE_scripts.py
Writing 's' (str) to file 'make_three_scripts.py'.
So we should have two scripts. Let's show those here.
!cat make_three_scripts.py
script_name = 'make_three_scripts.py'
import shortuuid
for x in range(1,4):
output_file_name_prefix = shortuuid.ShortUUID().random(length=10)
output_file_name = f"{output_file_name_prefix}.py"
so = f"print('I was output by script number {x} of 3 made by {script_name};"
so += f" generated by {output_file_name}')"
with open(output_file_name, 'w') as output_file:
output_file.write(so)
!cat make_three_MORE_scripts.py
script_name = 'make_three_MORE_scripts.py'
import shortuuid
for x in range(1,4):
output_file_name_prefix = shortuuid.ShortUUID().random(length=10)
output_file_name = f"{output_file_name_prefix}.py"
so = f"print('I was output by script number {x} of 3 made by {script_name};"
so += f" generated by {output_file_name}')"
with open(output_file_name, 'w') as output_file:
output_file.write(so)
Finally, having prepared everything, now we need to run each script and any Python scripts the scripts spawn once. This next part should cover what the OP sought.
Option to run each script once, allowing for additional Python scripts with the .py extension to also be generated in the course of running the scripts.
import os
import fnmatch
import glob
executed_scripts = []
extension_to_match = ".py"
def execute_script(s):
%run {s}
while set(executed_scripts) != set(glob.glob(f"*{extension_to_match}")):
for file in os.listdir('.'):
if fnmatch.fnmatch(file, '*'+extension_to_match):
if file not in executed_scripts:
execute_script(file)
executed_scripts.append(file)
print("\n\n\n**************------------DEMO COMPLETE------------------***************")
print(f"Number of scripts run in total: {len(executed_scripts)}")
scripts_spawned_dynamically_and_run = set(executed_scripts) - set(["make_three_scripts.py","make_three_MORE_scripts.py"])
print(f"Number of scripts spawn dynamically and run: {len(scripts_spawned_dynamically_and_run)}")
I was output by script number 2 of 3 made by make_three_MORE_scripts.py; generated by CwVVW2wWFx.py I was output by script number 3 of 3 made by make_three_scripts.py; generated by Ar2FAJMZ5Z.py I was output by script number 1 of 3 made by make_three_MORE_scripts.py; generated by HAenGmmo57.py I was output by script number 3 of 3 made by make_three_MORE_scripts.py; generated by 8gcHDrxzTp.py I was output by script number 2 of 3 made by make_three_scripts.py; generated by CfmcdhKqek.py I was output by script number 1 of 3 made by make_three_scripts.py; generated by AyUV8wpVTo.py **************------------DEMO COMPLETE------------------*************** Number of scripts run in total: 8 Number of scripts spawn dynamically and run: 6
Reset demo and try again? Uncomment and run the next cell to reset things by removing all the scripts.
#!rm -rf *.py
Re-comment that line in the cell above again so it will be inactive again, and then you'll be ready to use 'Kernel' > 'Restart Kernel and Run All Cells' to re-run from essentially scratch again. The pip install won't be repeated; howver, everything else will be re-run.
Enjoy!