
# 下载数据集摘要信息
!wget https://figshare.com/ndownloader/files/39406012 -O dataset_info.zip

--2023-02-27 17:32:47--  https://figshare.com/ndownloader/files/39406012
正在解析主机 figshare.com (figshare.com)... 54.217.34.18, 34.252.222.205, 2a05:d018:1f4:d003:825f:f38:d5f1:5837, ...
正在连接 figshare.com (figshare.com)|54.217.34.18|:443... 已连接。
已发出 HTTP 请求，正在等待回应... 302 Found
位置：https://s3-eu-west-1.amazonaws.com/pfigshare-u-files/39406012/nlfffdatasetinfo.zip?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIYCQYOYV5JSSROOA/20230227/eu-west-1/s3/aws4_request&X-Amz-Date=20230227T093248Z&X-Amz-Expires=10&X-Amz-SignedHeaders=host&X-Amz-Signature=6cc1ec50936034334e31d38e4dad769d4225553bea89128fd53512ab8b45f233 [跟随至新的 URL]
--2023-02-27 17:32:48--  https://s3-eu-west-1.amazonaws.com/pfigshare-u-files/39406012/nlfffdatasetinfo.zip?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIYCQYOYV5JSSROOA/20230227/eu-west-1/s3/aws4_request&X-Amz-Date=20230227T093248Z&X-Amz-Expires=10&X-Amz-SignedHeaders=host&X-Amz-Signature=6cc1ec50936034334e31d38e4dad769d4225553bea89128fd53512ab8b45f233
正在解析主机 s3-eu-west-1.amazonaws.com (s3-eu-west-1.amazonaws.com)... 52.218.37.83, 52.218.97.154, 52.218.53.162, ...
正在连接 s3-eu-west-1.amazonaws.com (s3-eu-west-1.amazonaws.com)|52.218.37.83|:443... 已连接。
已发出 HTTP 请求，正在等待回应... 200 OK
长度：166327650 (159M) [application/zip]
正在保存至: “dataset_info.zip”

dataset_info.zip    100%[===================>] 158.62M  1.50MB/s  用时 82s       

2023-02-27 17:34:12 (1.94 MB/s) - 已保存 “dataset_info.zip” [166327650/166327650])


# 解压数据集摘要信息
!unzip dataset_info.zip

Archive:  dataset_info.zip
   creating: nlfff-dataset-info/
  inflating: nlfff-dataset-info/README.txt  
   creating: nlfff-dataset-info/archive-202203/
  inflating: nlfff-dataset-info/archive-202203/archive-202203-info.csv  
  inflating: nlfff-dataset-info/changelog.txt  
  inflating: nlfff-dataset-info/config.json  
   creating: nlfff-dataset-info/demo/
   creating: nlfff-dataset-info/demo/hmi.sharp_cea_720s.7327.20181220_093600_TAI/
  inflating: nlfff-dataset-info/demo/hmi.sharp_cea_720s.7327.20181220_093600_TAI/Bout.bin  
  inflating: nlfff-dataset-info/demo/hmi.sharp_cea_720s.7327.20181220_093600_TAI/Energy.log  
  inflating: nlfff-dataset-info/demo/hmi.sharp_cea_720s.7327.20181220_093600_TAI/NLFFFquality1.log  
  inflating: nlfff-dataset-info/demo/hmi.sharp_cea_720s.7327.20181220_093600_TAI/NLFFFquality2.log  
  inflating: nlfff-dataset-info/demo/hmi.sharp_cea_720s.7327.20181220_093600_TAI/NLFFFquality3.log  
  inflating: nlfff-dataset-info/demo/hmi.sharp_cea_720s.7327.20181220_093600_TAI/allboundaries1.dat  
  inflating: nlfff-dataset-info/demo/hmi.sharp_cea_720s.7327.20181220_093600_TAI/allboundaries2.dat  
  inflating: nlfff-dataset-info/demo/hmi.sharp_cea_720s.7327.20181220_093600_TAI/allboundaries3.dat  
 extracting: nlfff-dataset-info/demo/hmi.sharp_cea_720s.7327.20181220_093600_TAI/boundary.ini  
 extracting: nlfff-dataset-info/demo/hmi.sharp_cea_720s.7327.20181220_093600_TAI/grid1.ini  
 extracting: nlfff-dataset-info/demo/hmi.sharp_cea_720s.7327.20181220_093600_TAI/grid2.ini  
  inflating: nlfff-dataset-info/demo/hmi.sharp_cea_720s.7327.20181220_093600_TAI/grid2n2.ini  
 extracting: nlfff-dataset-info/demo/hmi.sharp_cea_720s.7327.20181220_093600_TAI/grid3.ini  
  inflating: nlfff-dataset-info/demo/hmi.sharp_cea_720s.7327.20181220_093600_TAI/mask1.dat  
  inflating: nlfff-dataset-info/demo/hmi.sharp_cea_720s.7327.20181220_093600_TAI/mask2.dat  
  inflating: nlfff-dataset-info/demo/hmi.sharp_cea_720s.7327.20181220_093600_TAI/mask3.dat  
  inflating: nlfff-dataset-info/demo/hmi.sharp_cea_720s.7327.20181220_093600_TAI/prot1.log  
  inflating: nlfff-dataset-info/demo/hmi.sharp_cea_720s.7327.20181220_093600_TAI/prot2.log  
  inflating: nlfff-dataset-info/demo/hmi.sharp_cea_720s.7327.20181220_093600_TAI/prot3.log  
   creating: nlfff-dataset-info/demo/hmi.sharp_cea_720s.7327.20181220_093600_TAI/raw/
  inflating: nlfff-dataset-info/demo/hmi.sharp_cea_720s.7327.20181220_093600_TAI/raw/hmi.sharp_cea_720s.7327.20181220_093600_TAI.Bp.fits  
  inflating: nlfff-dataset-info/demo/hmi.sharp_cea_720s.7327.20181220_093600_TAI/raw/hmi.sharp_cea_720s.7327.20181220_093600_TAI.Br.fits  
  inflating: nlfff-dataset-info/demo/hmi.sharp_cea_720s.7327.20181220_093600_TAI/raw/hmi.sharp_cea_720s.7327.20181220_093600_TAI.Bt.fits  
  inflating: nlfff-dataset-info/demo/hmi.sharp_cea_720s.7327.20181220_093600_TAI/run.log  
 extracting: nlfff-dataset-info/demo/hmi.sharp_cea_720s.7327.20181220_093600_TAI/step1.log  
 extracting: nlfff-dataset-info/demo/hmi.sharp_cea_720s.7327.20181220_093600_TAI/step2.log  
 extracting: nlfff-dataset-info/demo/hmi.sharp_cea_720s.7327.20181220_093600_TAI/step3.log  
  inflating: nlfff-dataset-info/demo/hmi.sharp_cea_720s.7327.20181220_093600_TAI/zboundaries2n2.dat  
  inflating: nlfff-dataset-info/python_guide_en.ipynb  
  inflating: nlfff-dataset-info/python_guide_zh.ipynb  
   creating: nlfff-dataset-info/raw-202203/
  inflating: nlfff-dataset-info/raw-202203/raw-202203-info.csv


!wget https://figshare.com/ndownloader/files/39406015 -O dataset_label.zip

--2023-02-27 17:34:16--  https://figshare.com/ndownloader/files/39406015
正在解析主机 figshare.com (figshare.com)... 54.217.34.18, 34.252.222.205, 2a05:d018:1f4:d000:647c:7301:fad1:b2b9, ...
正在连接 figshare.com (figshare.com)|54.217.34.18|:443... 已连接。
已发出 HTTP 请求，正在等待回应... 302 Found
位置：https://s3-eu-west-1.amazonaws.com/pfigshare-u-files/39406015/nlfffdatasetflarelabel.zip?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIYCQYOYV5JSSROOA/20230227/eu-west-1/s3/aws4_request&X-Amz-Date=20230227T093417Z&X-Amz-Expires=10&X-Amz-SignedHeaders=host&X-Amz-Signature=1a0c58cc94e594610caf7e46ff979dbcfc4a764d8394cff00ccd5f52a06d1900 [跟随至新的 URL]
--2023-02-27 17:34:17--  https://s3-eu-west-1.amazonaws.com/pfigshare-u-files/39406015/nlfffdatasetflarelabel.zip?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIYCQYOYV5JSSROOA/20230227/eu-west-1/s3/aws4_request&X-Amz-Date=20230227T093417Z&X-Amz-Expires=10&X-Amz-SignedHeaders=host&X-Amz-Signature=1a0c58cc94e594610caf7e46ff979dbcfc4a764d8394cff00ccd5f52a06d1900
正在解析主机 s3-eu-west-1.amazonaws.com (s3-eu-west-1.amazonaws.com)... 52.218.100.83, 52.218.96.18, 52.218.89.19, ...
正在连接 s3-eu-west-1.amazonaws.com (s3-eu-west-1.amazonaws.com)|52.218.100.83|:443... 已连接。
已发出 HTTP 请求，正在等待回应... 200 OK
长度：3249636 (3.1M) [application/zip]
正在保存至: “dataset_label.zip”

dataset_label.zip   100%[===================>]   3.10M   747KB/s  用时 4.8s      

2023-02-27 17:34:23 (657 KB/s) - 已保存 “dataset_label.zip” [3249636/3249636])


!unzip dataset_label.zip

Archive:  dataset_label.zip
   creating: nlfff-dataset-flare-label/
  inflating: nlfff-dataset-flare-label/README.txt  
  inflating: nlfff-dataset-flare-label/changelog.txt  
  inflating: nlfff-dataset-flare-label/config.json  
   creating: nlfff-dataset-flare-label/label-202203/
   creating: nlfff-dataset-flare-label/label-202203/prepare/
  inflating: nlfff-dataset-flare-label/label-202203/prepare/knoaa_vflaretimelist.pickle  
  inflating: nlfff-dataset-flare-label/label-202203/prepare/ksharp_vnoaa.pickle  
   creating: nlfff-dataset-flare-label/label-202203/process/
  inflating: nlfff-dataset-flare-label/label-202203/process/label.csv  
  inflating: nlfff-dataset-flare-label/label-202203/process/label_col_description.jpg  
   creating: nlfff-dataset-flare-label/label-202203/process/label_failed/
  inflating: nlfff-dataset-flare-label/label-202203/process/label_failed/no_flare_info_from_noaa_ars.txt  
  inflating: nlfff-dataset-flare-label/label-202203/process/label_failed/no_noaalist_from_sharpnum.txt  
   creating: nlfff-dataset-flare-label/label-202203/raw_file/
  inflating: nlfff-dataset-flare-label/label-202203/raw_file/all_harps_with_noaa_ars.txt  
  inflating: nlfff-dataset-flare-label/label-202203/raw_file/flare_info_processed.csv  
  inflating: nlfff-dataset-flare-label/label-202203/raw_file/flare_info_processed_col_description.jpg  
  inflating: nlfff-dataset-flare-label/python_guide_en.ipynb  
  inflating: nlfff-dataset-flare-label/python_guide_zh.ipynb


!wget https://figshare.com/ndownloader/files/39406063 -O dataset_archive.zip

--2023-02-27 17:34:24--  https://figshare.com/ndownloader/files/39406063
正在解析主机 figshare.com (figshare.com)... 34.252.222.205, 54.217.34.18, 2a05:d018:1f4:d003:825f:f38:d5f1:5837, ...
正在连接 figshare.com (figshare.com)|34.252.222.205|:443... 已连接。
已发出 HTTP 请求，正在等待回应... 302 Found
位置：https://s3-eu-west-1.amazonaws.com/pfigshare-u-files/39406063/nlfffdatasetandflarelabeldbarchive.zip?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIYCQYOYV5JSSROOA/20230227/eu-west-1/s3/aws4_request&X-Amz-Date=20230227T093425Z&X-Amz-Expires=10&X-Amz-SignedHeaders=host&X-Amz-Signature=d2c1c98b7e21b7d3273e9a876e6764e6a17693538d40dd6f23fdfc90ad4fbd39 [跟随至新的 URL]
--2023-02-27 17:34:25--  https://s3-eu-west-1.amazonaws.com/pfigshare-u-files/39406063/nlfffdatasetandflarelabeldbarchive.zip?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIYCQYOYV5JSSROOA/20230227/eu-west-1/s3/aws4_request&X-Amz-Date=20230227T093425Z&X-Amz-Expires=10&X-Amz-SignedHeaders=host&X-Amz-Signature=d2c1c98b7e21b7d3273e9a876e6764e6a17693538d40dd6f23fdfc90ad4fbd39
正在解析主机 s3-eu-west-1.amazonaws.com (s3-eu-west-1.amazonaws.com)... 52.218.112.11, 52.218.100.83, 52.218.89.19, ...
正在连接 s3-eu-west-1.amazonaws.com (s3-eu-west-1.amazonaws.com)|52.218.112.11|:443... 已连接。
已发出 HTTP 请求，正在等待回应... 200 OK
长度：85650805 (82M) [application/zip]
正在保存至: “dataset_archive.zip”

dataset_archive.zip 100%[===================>]  81.68M  2.42MB/s  用时 31s       

2023-02-27 17:34:57 (2.65 MB/s) - 已保存 “dataset_archive.zip” [85650805/85650805])


!unzip dataset_archive.zip

Archive:  dataset_archive.zip
   creating: nlfff-dataset-and-flare-label-db-archive/
  inflating: nlfff-dataset-and-flare-label-db-archive/README.txt  
  inflating: nlfff-dataset-and-flare-label-db-archive/changelog.txt  
  inflating: nlfff-dataset-and-flare-label-db-archive/config.json  
  inflating: nlfff-dataset-and-flare-label-db-archive/python_guide_en.ipynb  
  inflating: nlfff-dataset-and-flare-label-db-archive/python_guide_zh.ipynb  
   creating: nlfff-dataset-and-flare-label-db-archive/sqlite-202203/
  inflating: nlfff-dataset-and-flare-label-db-archive/sqlite-202203/nlfff-202203


nlfff_path_dataset_summary="nlfff-dataset-info"
nlfff_path_dataset_label="nlfff-dataset-flare-label"
nlfff_path_dataset_archive="nlfff-dataset-and-flare-label-db-archive"


import os
import json

# 读取配置文件
nlfff_path_dataset_summary="nlfff-dataset-info"
config_path=os.path.join(nlfff_path_dataset_summary,"config.json")
with open(config_path,"r") as f:
    config_ds=json.loads(f.read())


config_ds

{'config_version': 1.3,
 'raw_summary_info_path': 'raw-202203/raw-202203-info.csv',
 'product_summary_info_path': 'archive-202203/archive-202203-info.csv',
 'demo_sample': [{'harpnum_trec': '7327.20181220_093600_TAI',
   'path': 'demo/hmi.sharp_cea_720s.7327.20181220_093600_TAI'}]}


import pandas
import os

raw_info_path=os.path.join(
  nlfff_path_dataset_summary,
  config_ds["raw_summary_info_path"]
)
raw_info=pandas.read_csv(raw_info_path)

/home/zander/anaconda3/lib/python3.9/site-packages/IPython/core/interactiveshell.py:3444: DtypeWarning: Columns (145,254) have mixed types.Specify dtype option on import or set low_memory=False.
  exec(code_obj, self.user_global_ns, self.user_ns)


raw_info


product_info_path=os.path.join(
  nlfff_path_dataset_summary,
  config_ds["product_summary_info_path"]
)
product_info=pandas.read_csv(product_info_path)


product_info


import os
import json

# 读取配置文件
nlfff_path_dataset_label="nlfff-dataset-flare-label"
config_path=os.path.join(nlfff_path_dataset_label,"config.json")
with open(config_path,"r") as f:
    config_label=json.loads(f.read())


config_label

{'config_version': 1.3,
 'flare_info_path': 'label-202203/raw_file/flare_info_processed.csv',
 'prepare_file': {'knoaa_vflaretimelist_path': 'label-202203/prepare/knoaa_vflaretimelist.pickle',
  'ksharp_vnoaa_path': 'label-202203/prepare/ksharp_vnoaa.pickle'},
 'label_info_path': 'label-202203/process/label.csv'}


label_info_path=os.path.join(
    nlfff_path_dataset_label,
    config_label["label_info_path"]
)
label_info=pandas.read_csv(label_info_path)


label_info


flare_info_path=os.path.join(
    nlfff_path_dataset_label,
    config_label["flare_info_path"]
)

flare_info=pandas.read_csv(flare_info_path)


flare_info


import json
import sqlite3
import os


import os
import json

# 读取配置文件
nlfff_path_dataset_archive="nlfff-dataset-and-flare-label-db-archive"
config_path=os.path.join(nlfff_path_dataset_archive,"config.json")
with open(config_path,"r") as f:
    config_db=json.loads(f.read())


config_db

{'config_version': 1.3, 'archive_db_path': 'sqlite-202203/nlfff-202203'}


archive_db_path=os.path.join(
    nlfff_path_dataset_archive,
    config_db["archive_db_path"]
)
archive_db_path

'nlfff-dataset-and-flare-label-db-archive/sqlite-202203/nlfff-202203'


# 查询存在哪些表
query_sql="select name from sqlite_master where type='table' order by name; "
table_list=[]
with sqlite3.connect(archive_db_path) as conn:
   c = conn.cursor()
   cursor = c.execute(query_sql)
   for table in cursor:
      print(table)
      table_list.append(table[0])
print(table_list)

('flare_info',)
('nlfff_archive',)
('nlfff_flare_label',)
('nlfff_raw',)
['flare_info', 'nlfff_archive', 'nlfff_flare_label', 'nlfff_raw']


# 查询某个表有哪些列
table_name="nlfff_archive"
query_sql="PRAGMA table_info({});".format(table_name)
column_list=[]
with sqlite3.connect(archive_db_path) as conn:
   c = conn.cursor()
   cursor = c.execute(query_sql)
   for column in cursor:
      print(column)
      column_list.append(column[1])
print(column_list)

(0, 'harpnum_trec', 'VARCHAR(50)', 0, None, 0)
(1, 'sharp_num', 'INTEGER', 0, None, 0)
(2, 't_rec', 'VARCHAR(50)', 0, None, 0)
(3, 'bout_path', 'VARCHAR(128)', 0, None, 0)
(4, 'bout_md5', 'VARCHAR(50)', 0, None, 0)
(5, 'bout_size', 'INTEGER', 0, None, 0)
(6, 'grid_x', 'INTEGER', 0, None, 0)
(7, 'grid_y', 'INTEGER', 0, None, 0)
(8, 'grid_z', 'INTEGER', 0, None, 0)
(9, 'grid_xyz', 'INTEGER', 0, None, 0)
(10, 'bout_maxlevel', 'INTEGER', 0, None, 0)
(11, 'bout_quality', 'BOOLEAN', 0, None, 0)
(12, 'batch', 'INTEGER', 0, None, 0)
(13, 'notes', 'VARCHAR(50)', 0, None, 0)
(14, 'bout_quality_value', 'REAL', 0, None, 0)
(15, 'identifiers', 'VARCHAR(50)', 0, None, 0)
['harpnum_trec', 'sharp_num', 't_rec', 'bout_path', 'bout_md5', 'bout_size', 'grid_x', 'grid_y', 'grid_z', 'grid_xyz', 'bout_maxlevel', 'bout_quality', 'batch', 'notes', 'bout_quality_value', 'identifiers']


# 创建表名和列名的映射字典

ktablename_vcolumnlist=dict()

for table_name in table_list:
   print(table_name)
   # table_name="nlfff_archive"
   query_sql="PRAGMA table_info({});".format(table_name)
   column_list=[]
   with sqlite3.connect(archive_db_path) as conn:
      c = conn.cursor()
      cursor = c.execute(query_sql)
      for column in cursor:
         # print(column)
         column_list.append(column[1])
   # print(column_list)
   ktablename_vcolumnlist[table_name]=column_list

flare_info
nlfff_archive
nlfff_flare_label
nlfff_raw


ktablename_vcolumnlist

{'flare_info': ['deeps_flare_id',
  'start_datetime',
  'end_datetime',
  'peak_datetime',
  'xray_class',
  'xray_intensity',
  'latitude',
  'longtitude',
  'noaa_ar',
  'source'],
 'nlfff_archive': ['harpnum_trec',
  'sharp_num',
  't_rec',
  'bout_path',
  'bout_md5',
  'bout_size',
  'grid_x',
  'grid_y',
  'grid_z',
  'grid_xyz',
  'bout_maxlevel',
  'bout_quality',
  'batch',
  'notes',
  'bout_quality_value',
  'identifiers'],
 'nlfff_flare_label': ['harpnum_trec',
  'harp_num',
  't_rec_str',
  'now_flare_level',
  'now_flare_id',
  'h6_flare_level',
  'h6_flare_id',
  'h12_flare_level',
  'h12_flare_id',
  'h24_flare_level',
  'h24_flare_id',
  'h48_flare_level',
  'h48_flare_id',
  't_rec_datetime',
  'h24_posmx',
  'h24_poscmx',
  'h48_posmx',
  'h48_poscmx',
  'h24_delta05',
  'h48_delta05'],
 'nlfff_raw': ['harpnum_trec',
  'HARP_NUM',
  'T_REC_DATETIME',
  'T_REC_STR',
  'FILE_NAME',
  'query',
  'DATE',
  'DATE_S',
  'DATE_B',
  'DATE__OBS',
  'T_OBS',
  'T_REC',
  'T_REC_epoch',
  'T_REC_step',
  'T_REC_unit',
  'CADENCE',
  'USFLUX',
  'MEANGAM',
  'MEANGBT',
  'MEANGBZ',
  'MEANGBH',
  'MEANJZD',
  'TOTUSJZ',
  'MEANALP',
  'MEANJZH',
  'TOTUSJH',
  'ABSNJZH',
  'SAVNCPP',
  'MEANPOT',
  'TOTPOT',
  'MEANSHR',
  'SHRGT45',
  'R_VALUE',
  'GWILL',
  'CTYPE1',
  'CTYPE2',
  'CRPIX1',
  'CRPIX2',
  'CRVAL1',
  'CRVAL2',
  'CDELT1',
  'CDELT2',
  'CUNIT1',
  'CUNIT2',
  'IMCRPIX1',
  'IMCRPIX2',
  'IMCRVAL1',
  'IMCRVAL2',
  'CROTA2',
  'CRDER1',
  'CRDER2',
  'CSYSER1',
  'CSYSER2',
  'WCSNAME',
  'DSUN_OBS',
  'DSUN_REF',
  'RSUN_REF',
  'CRLN_OBS',
  'CRLT_OBS',
  'CAR_ROT',
  'OBS_VR',
  'OBS_VW',
  'OBS_VN',
  'RSUN_OBS',
  'TELESCOP',
  'INSTRUME',
  'WAVELNTH',
  'CAMERA',
  'QUALITY',
  'QUAL_S',
  'QUALLEV1',
  'BUNIT_000',
  'BUNIT_001',
  'BUNIT_002',
  'BUNIT_003',
  'BUNIT_004',
  'BUNIT_005',
  'BUNIT_006',
  'BUNIT_007',
  'BUNIT_008',
  'BUNIT_009',
  'BUNIT_010',
  'ORIGIN',
  'CONTENT',
  'HISTORY',
  'COMMENT',
  'BLD_VERS',
  'CALVER64',
  'CODEVER7',
  'HFLID',
  'HCFTID',
  'QLOOK',
  'HARPNUM',
  'MASK',
  'ARM_QUAL',
  'ARM_NCLN',
  'H_MERGE',
  'H_FAINT',
  'ARM_MODL',
  'ARM_EDGE',
  'ARM_BETA',
  'LATDTMIN',
  'LONDTMIN',
  'LATDTMAX',
  'LONDTMAX',
  'OMEGA_DT',
  'NPIX',
  'SIZE',
  'AREA',
  'NACR',
  'SIZE_ACR',
  'AREA_ACR',
  'MTOT',
  'MNET',
  'MPOS_TOT',
  'MNEG_TOT',
  'MMEAN',
  'MSTDEV',
  'MSKEW',
  'MKURT',
  'LAT_MIN',
  'LON_MIN',
  'LAT_MAX',
  'LON_MAX',
  'LAT_FWT',
  'LON_FWT',
  'LAT_FWTPOS',
  'LON_FWTPOS',
  'LAT_FWTNEG',
  'LON_FWTNEG',
  'T_FRST',
  'T_FRST1',
  'T_LAST1',
  'T_LAST',
  'N_PATCH',
  'N_PATCH1',
  'N_PATCHM',
  'NOAA_AR',
  'NOAA_NUM',
  'NOAA_ARS',
  'INVCODEV',
  'INVDOCU',
  'INVITERA',
  'INVSTLGT',
  'INVFLPRF',
  'INVPHMAP',
  'INVVLAVE',
  'INVBLAVE',
  'INVBBAVE',
  'INVNPRCS',
  'INVNCNVG',
  'AMBCODEV',
  'AMBDOCU',
  'AMBGMTRY',
  'AMBPATCH',
  'AMBWEAK',
  'AMBNEROD',
  'AMBNGROW',
  'AMBNPAD',
  'AMBNAP',
  'AMBNTX',
  'AMBNTY',
  'AMBBTHR0',
  'AMBBTHR1',
  'AMBSEED',
  'AMBNEQ',
  'AMBLMBDA',
  'AMBTFCT0',
  'AMBTFCTR',
  'DATAVALS_000',
  'DATAVALS_001',
  'DATAVALS_002',
  'DATAVALS_003',
  'DATAVALS_004',
  'DATAVALS_005',
  'DATAVALS_006',
  'DATAVALS_007',
  'DATAVALS_008',
  'DATAVALS_009',
  'DATAVALS_010',
  'MISSVALS_000',
  'MISSVALS_001',
  'MISSVALS_002',
  'MISSVALS_003',
  'MISSVALS_004',
  'MISSVALS_005',
  'MISSVALS_006',
  'MISSVALS_007',
  'MISSVALS_008',
  'MISSVALS_009',
  'MISSVALS_010',
  'DATAMIN_001',
  'DATAMIN_002',
  'DATAMIN_003',
  'DATAMIN_004',
  'DATAMIN_005',
  'DATAMIN_006',
  'DATAMIN_007',
  'DATAMIN_008',
  'DATAMIN_009',
  'DATAMIN_010',
  'DATAMIN_000',
  'DATAMAX_000',
  'DATAMAX_001',
  'DATAMAX_002',
  'DATAMAX_003',
  'DATAMAX_004',
  'DATAMAX_005',
  'DATAMAX_006',
  'DATAMAX_007',
  'DATAMAX_008',
  'DATAMAX_009',
  'DATAMAX_010',
  'DATAMEDN_003',
  'DATAMEDN_004',
  'DATAMEDN_005',
  'DATAMEDN_006',
  'DATAMEDN_007',
  'DATAMEDN_008',
  'DATAMEDN_009',
  'DATAMEDN_010',
  'DATAMEDN_000',
  'DATAMEDN_001',
  'DATAMEDN_002',
  'DATAMEAN_010',
  'DATAMEAN_000',
  'DATAMEAN_001',
  'DATAMEAN_002',
  'DATAMEAN_003',
  'DATAMEAN_004',
  'DATAMEAN_005',
  'DATAMEAN_006',
  'DATAMEAN_007',
  'DATAMEAN_008',
  'DATAMEAN_009',
  'DATARMS_000',
  'DATARMS_001',
  'DATARMS_002',
  'DATARMS_003',
  'DATARMS_004',
  'DATARMS_005',
  'DATARMS_006',
  'DATARMS_007',
  'DATARMS_008',
  'DATARMS_009',
  'DATARMS_010',
  'ERRGAM',
  'ERRTAI',
  'ERRBH',
  'ERRMPOT',
  'ERRBT',
  'ERRTUI',
  'ERRBZ',
  'CMASK',
  'ERRJZ',
  'ERRVF',
  'ERRALP',
  'ERRMIH',
  'ERRMSHA',
  'ERRUSI',
  'DOFFSET',
  'ERRTPOT',
  'ERRJHT',
  'magnetogram',
  'bitmap',
  'Dopplergram',
  'continuum',
  'Bp',
  'Bt',
  'Br',
  'Bp_err',
  'Bt_err',
  'Br_err',
  'conf_disambig',
  'BHARP',
  'MHARP']}


# 样本原始fits文件头文件信息(time,noaa number)

table_name="nlfff_raw"
with sqlite3.connect(archive_db_path) as conn:
   c = conn.cursor()

   query_sql="SELECT *  from {}".format(table_name)
   cursor = c.execute(query_sql)
   for row in cursor:
      row_data_list=list(row)
      row_column_name_list=ktablename_vcolumnlist[table_name]
      row_dict=dict(zip(row_column_name_list,row_data_list))
      print(row_dict)
      break

{'harpnum_trec': '1644.20120516_031200_TAI', 'HARP_NUM': 1644, 'T_REC_DATETIME': '2012-05-16 03:12:00.000', 'T_REC_STR': '20120516_031200_TAI', 'FILE_NAME': 'hmi.sharp_cea_720s.1644.20120516_031200_TAI', 'query': 'hmi.sharp_cea_720s[1644][2012.05.16_03:12:00_TAI]', 'DATE': '2018-05-09T03:05:47Z', 'DATE_S': '2012-05-20T19:22:46Z', 'DATE_B': '2015-09-30T20:47:04Z', 'DATE__OBS': '2012-05-16T03:10:24.10Z', 'T_OBS': '2012.05.16_03:12:06_TAI', 'T_REC': '2012.05.16_03:12:00_TAI', 'T_REC_epoch': '1993.01.01_00:00:00_TAI', 'T_REC_step': 720.0, 'T_REC_unit': 'secs', 'CADENCE': 720.0, 'USFLUX': 7.195475e+21, 'MEANGAM': 47.02, 'MEANGBT': 113.15, 'MEANGBZ': 110.28299999999999, 'MEANGBH': 61.652, 'MEANJZD': 0.12072397, 'TOTUSJZ': 9593281000000, 'MEANALP': 0.01153706, 'MEANJZH': 0.0052498, 'TOTUSJH': 474.784, 'ABSNJZH': 57.433, 'SAVNCPP': 710574700000, 'MEANPOT': 9327.876, 'TOTPOT': 1.355353e+23, 'MEANSHR': 36.965, 'SHRGT45': 33.726, 'R_VALUE': 2.103, 'GWILL': '', 'CTYPE1': 'CRLN-CEA', 'CTYPE2': 'CRLT-CEA', 'CRPIX1': 477.5, 'CRPIX2': 279.0, 'CRVAL1': 134.58140600000002, 'CRVAL2': -22.961349, 'CDELT1': 0.03, 'CDELT2': 0.03, 'CUNIT1': 'degree', 'CUNIT2': 'degree', 'IMCRPIX1': 2035.023315, 'IMCRPIX2': 2052.667725, 'IMCRVAL1': 0.0, 'IMCRVAL2': 0.0, 'CROTA2': 0.0, 'CRDER1': '', 'CRDER2': '', 'CSYSER1': '', 'CSYSER2': '', 'WCSNAME': 'Carrington Heliographic', 'DSUN_OBS': 151288617876.01, 'DSUN_REF': 149597870691, 'RSUN_REF': 696000000, 'CRLN_OBS': 122.84070600000001, 'CRLT_OBS': -2.505391, 'CAR_ROT': 2123, 'OBS_VR': 1939.0652760000003, 'OBS_VW': 30084.039826, 'OBS_VN': 6001.302302, 'RSUN_OBS': 948.920105, 'TELESCOP': 'SDO/HMI', 'INSTRUME': 'HMI_SIDE1', 'WAVELNTH': 6173.0, 'CAMERA': 1, 'QUALITY': '0x00000000', 'QUAL_S': '0x00000000', 'QUALLEV1': '0x00000000', 'BUNIT_000': 'Mx/cm^2', 'BUNIT_001': ' ', 'BUNIT_002': 'cm/s', 'BUNIT_003': 'DN/s', 'BUNIT_004': 'Mx/cm^2', 'BUNIT_005': 'Mx/cm^2', 'BUNIT_006': 'Mx/cm^2', 'BUNIT_007': 'Mx/cm^2', 'BUNIT_008': 'Mx/cm^2', 'BUNIT_009': 'Mx/cm^2', 'BUNIT_010': ' ', 'ORIGIN': 'SDO/JSOC-SDP', 'CONTENT': 'HMI observable', 'HISTORY': ' ', 'COMMENT': ' ', 'BLD_VERS': 'V8R9X', 'CALVER64': '0x0000000000000010', 'CODEVER7': '$Id: sharp.c,v 1.38 2015/03/18 00:28:26 xudong Exp $', 'HFLID': 1021, 'HCFTID': 11, 'QLOOK': 0, 'HARPNUM': 1644, 'MASK': 32, 'ARM_QUAL': '0x00000000', 'ARM_NCLN': 1, 'H_MERGE': 0, 'H_FAINT': 0, 'ARM_MODL': '/builtin/hmi.M_Ic_noLimbDark_720s.production', 'ARM_EDGE': 2.5, 'ARM_BETA': 0.3, 'LATDTMIN': -31.313299, 'LONDTMIN': -2.5699, 'LATDTMAX': -14.6094, 'LONDTMAX': 26.0513, 'OMEGA_DT': 13.2649, 'NPIX': 133203, 'SIZE': 11976.625, 'AREA': 6635.683594, 'NACR': 9287, 'SIZE_ACR': 835.018066, 'AREA_ACR': 462.282288, 'MTOT': 6930884.5, 'MNET': 260587.09375, 'MPOS_TOT': 3595736.0, 'MNEG_TOT': 3335148.75, 'MMEAN': 1.956316, 'MSTDEV': 144.275909, 'MSKEW': 4.339589, 'MKURT': 41.763123, 'LAT_MIN': -30.760214, 'LON_MIN': -1.510393, 'LAT_MAX': -15.120847, 'LON_MAX': 23.849808, 'LAT_FWT': -23.853493, 'LON_FWT': 12.266521000000001, 'LAT_FWTPOS': -23.781185, 'LON_FWTPOS': 14.561793, 'LAT_FWTNEG': -23.969935999999997, 'LON_FWTNEG': 8.570277, 'T_FRST': '2012.05.06_21:12:00_TAI', 'T_FRST1': '2012.05.07_21:12:00_TAI', 'T_LAST1': '2012.05.22_00:48:00_TAI', 'T_LAST': '2012.05.23_00:48:00_TAI', 'N_PATCH': 1939, 'N_PATCH1': 1699, 'N_PATCHM': 0, 'NOAA_AR': 11477, 'NOAA_NUM': 2, 'NOAA_ARS': '11477,11478', 'INVCODEV': 'vfisv FD10 2012 Apr. 13', 'INVDOCU': '', 'INVITERA': 200, 'INVSTLGT': 'No', 'INVFLPRF': ' ', 'INVPHMAP': ' ', 'INVVLAVE': 1.033407e+22, 'INVBLAVE': 20.044, 'INVBBAVE': 127.6648, 'INVNPRCS': 150736, 'INVNCNVG': 11121934, 'AMBCODEV': 'disambig_v3 2013 Dec 06', 'AMBDOCU': '', 'AMBGMTRY': 2, 'AMBPATCH': 0, 'AMBWEAK': 2, 'AMBNEROD': 1, 'AMBNGROW': 5, 'AMBNPAD': 200, 'AMBNAP': 10, 'AMBNTX': 30, 'AMBNTY': 30, 'AMBBTHR0': 200.0, 'AMBBTHR1': 400.0, 'AMBSEED': 4, 'AMBNEQ': 100, 'AMBLMBDA': 1.0, 'AMBTFCT0': 2.0, 'AMBTFCTR': 0.98, 'DATAVALS_000': 531378, 'DATAVALS_001': 531378, 'DATAVALS_002': 531378, 'DATAVALS_003': 531378, 'DATAVALS_004': 531378, 'DATAVALS_005': 531378, 'DATAVALS_006': 531378, 'DATAVALS_007': 531378, 'DATAVALS_008': 531378, 'DATAVALS_009': 531378, 'DATAVALS_010': 531378, 'MISSVALS_000': 0, 'MISSVALS_001': 0, 'MISSVALS_002': 0, 'MISSVALS_003': 0, 'MISSVALS_004': 0, 'MISSVALS_005': 0, 'MISSVALS_006': 0, 'MISSVALS_007': 0, 'MISSVALS_008': 0, 'MISSVALS_009': 0, 'MISSVALS_010': 0, 'DATAMIN_001': 782.0, 'DATAMIN_002': 7730.0, 'DATAMIN_003': -1080.859985, 'DATAMIN_004': -1407.140015, 'DATAMIN_005': -1220.349976, 'DATAMIN_006': 5.495, 'DATAMIN_007': 5.89, 'DATAMIN_008': 3.056, 'DATAMIN_009': 50.0, 'DATAMIN_010': -883.099976, 'DATAMIN_000': 0.0, 'DATAMAX_000': 1839.199951, 'DATAMAX_001': 34.0, 'DATAMAX_002': 3945.0, 'DATAMAX_003': 64898.0, 'DATAMAX_004': 1343.079956, 'DATAMAX_005': 1352.119995, 'DATAMAX_006': 2752.580078, 'DATAMAX_007': 1515.130005, 'DATAMAX_008': 639.731018, 'DATAMAX_009': 361.239014, 'DATAMAX_010': 90.0, 'DATAMEDN_003': -4.422134, 'DATAMEDN_004': 41.280186, 'DATAMEDN_005': 40.248523999999996, 'DATAMEDN_006': 18.449409, 'DATAMEDN_007': 50.000305, 'DATAMEDN_008': -0.7078, 'DATAMEDN_009': 0.999985, 'DATAMEDN_010': 2272.499023, 'DATAMEDN_000': 56360.22265599999, 'DATAMEDN_001': 4.672016999999999, 'DATAMEDN_002': -9.005756, 'DATAMEAN_010': 20.131287, 'DATAMEAN_000': 52.015175, 'DATAMEAN_001': 0.475767, 'DATAMEAN_002': 10.057181, 'DATAMEAN_003': 2277.6877440000003, 'DATAMEAN_004': 56307.226562, 'DATAMEAN_005': 5.432481, 'DATAMEAN_006': -6.025362, 'DATAMEAN_007': 1.694795, 'DATAMEAN_008': 44.34304, 'DATAMEAN_009': 43.243214, 'DATARMS_000': 80.526901, 'DATARMS_001': 14.483751000000002, 'DATARMS_002': 386.348206, 'DATARMS_003': 2445.601807, 'DATARMS_004': 77.775124, 'DATARMS_005': 82.911629, 'DATARMS_006': 115.430428, 'DATARMS_007': 16.401438, 'DATARMS_008': 15.701928, 'DATARMS_009': 9.27594, 'DATARMS_010': 6.639278999999999, 'ERRGAM': 0.033, 'ERRTAI': 7.831, 'ERRBH': 0.237, 'ERRMPOT': 0.1888691, 'ERRBT': 10.901, 'ERRTUI': 0.16, 'ERRBZ': 7.831, 'CMASK': 10941.0, 'ERRJZ': 3261369466163822100, 'ERRVF': 0.07241935, 'ERRALP': 0.00157312, 'ERRMIH': 0.00071585, 'ERRMSHA': 0.003, 'ERRUSI': 105226300000, 'DOFFSET': 1054507008, 'ERRTPOT': 1.58388e+20, 'ERRJHT': 50.0, 'magnetogram': '/SUM90/D1042885471/S00000/magnetogram.fits', 'bitmap': '/SUM90/D1042885471/S00000/bitmap.fits', 'Dopplergram': '/SUM90/D1042885471/S00000/Dopplergram.fits', 'continuum': '/SUM90/D1042885471/S00000/continuum.fits', 'Bp': '/SUM90/D1042885471/S00000/Bp.fits', 'Bt': '/SUM90/D1042885471/S00000/Bt.fits', 'Br': '/SUM90/D1042885471/S00000/Br.fits', 'Bp_err': '/SUM90/D1042885471/S00000/Bp_err.fits', 'Bt_err': '/SUM90/D1042885471/S00000/Bt_err.fits', 'Br_err': '/SUM90/D1042885471/S00000/Br_err.fits', 'conf_disambig': '/SUM90/D1042885471/S00000/conf_disambig.fits', 'BHARP': 'InvalidLink', 'MHARP': 'hmi.Mharp_720s[1644][2012.05.16_03:12:00_TAI]'}


# 样本的产品信息（nx,ny,nz）

table_name="nlfff_archive"
with sqlite3.connect(archive_db_path) as conn:
   c = conn.cursor()

   query_sql="SELECT *  from {}".format(table_name)
   cursor = c.execute(query_sql)
   for row in cursor:
      row_data_list=list(row)
      row_column_name_list=ktablename_vcolumnlist[table_name]
      row_dict=dict(zip(row_column_name_list,row_data_list))
      print(row_dict)
      break

{'harpnum_trec': '6558.20160520_044800_TAI', 'sharp_num': 6558, 't_rec': '2016-05-20 04:48:00.000', 'bout_path': 'grid3.done/num_6500_6599/hmi.sharp_cea_720s.6558.20160520_044800_TAI/Bout.bin', 'bout_md5': 'ea4c960822a1284fe43637b1717bc30b', 'bout_size': 3580360704, 'grid_x': 432, 'grid_y': 764, 'grid_z': 452, 'grid_xyz': 149181696, 'bout_maxlevel': 3, 'bout_quality': 'true', 'batch': 20220301, 'notes': 'new', 'bout_quality_value': 15.568, 'identifiers': '6558201605200448003'}


# 样本标签（CMX flare label）

table_name="nlfff_flare_label"
with sqlite3.connect(archive_db_path) as conn:
   c = conn.cursor()

   query_sql="SELECT *  from {}".format(table_name)
   cursor = c.execute(query_sql)
   for row in cursor:
      row_data_list=list(row)
      row_column_name_list=ktablename_vcolumnlist[table_name]
      row_dict=dict(zip(row_column_name_list,row_data_list))
      print(row_dict)
      break

{'harpnum_trec': '1021.20111031_093600_TAI', 'harp_num': 1021, 't_rec_str': '20111031_093600_TAI', 'now_flare_level': 0, 'now_flare_id': -1, 'h6_flare_level': 0, 'h6_flare_id': -1, 'h12_flare_level': 0, 'h12_flare_id': -1, 'h24_flare_level': 0, 'h24_flare_id': -1, 'h48_flare_level': 0, 'h48_flare_id': -1, 't_rec_datetime': '2011-10-31 09:36:00.000', 'h24_posmx': 0, 'h24_poscmx': 0, 'h48_posmx': 0, 'h48_poscmx': 0, 'h24_delta05': 0, 'h48_delta05': 0}


# 耀斑信息（start time）

table_name="flare_info"
with sqlite3.connect(archive_db_path) as conn:
   c = conn.cursor()

   query_sql="SELECT *  from {}".format(table_name)
   cursor = c.execute(query_sql)
   for row in cursor:
      row_data_list=list(row)
      row_column_name_list=ktablename_vcolumnlist[table_name]
      row_dict=dict(zip(row_column_name_list,row_data_list))
      print(row_dict)
      break

{'deeps_flare_id': 78524, 'start_datetime': '2016-01-01 23:10:00.000', 'end_datetime': '2016-01-02 01:01:00.000', 'peak_datetime': '2016-01-02 00:11:00.000', 'xray_class': 'M', 'xray_intensity': 23, 'latitude': '', 'longtitude': '', 'noaa_ar': 12473, 'source': 'swpc'}


# 在数据集摘要信息里面有大于或等于一个样本，
# 我们以这个样本为例进行说明

# 以0号样本为例，读取已经下载样本的`harpnum_trec`和路径
nlfff_path_dataset_summary="nlfff-dataset-info"
sample_num=0
sample_harpnum_trec=config_ds["demo_sample"][sample_num]["harpnum_trec"]
sample_path=os.path.join(
    nlfff_path_dataset_summary,
    config_ds["demo_sample"][sample_num]["path"])


# 样本的harp number 和时间
sample_harpnum_trec

'7327.20181220_093600_TAI'


# Bout path
sample_bout_path=os.path.join(sample_path,"Bout.bin")
sample_bout_path

'nlfff-dataset-info/demo/hmi.sharp_cea_720s.7327.20181220_093600_TAI/Bout.bin'


# 获取样本产品文件信息

sample_product_info=product_info[ product_info["harpnum_trec"]==sample_harpnum_trec ]


sample_product_info


# 样本的最大计算等级 
sample_bout_maxlevel=int(sample_product_info["bout_maxlevel"])
# 样本的nx，ny，nz分别为
sample_nx=int(sample_product_info["grid_x"])
sample_ny=int(sample_product_info["grid_y"])
sample_nz=int(sample_product_info["grid_z"])
# 样本的identifiers为
sample_identifiers=int(sample_product_info["identifiers"])


# 根据 nx ny nz 读取数据
import numpy

nx=sample_nx
ny=sample_ny
nz=sample_nz

np_dtype_str=r"<d"
bin_path=os.path.join(sample_path,"Bout.bin")

# https://numpy.org/doc/stable/reference/generated/numpy.memmap.html
nlfff_data = numpy.memmap(sample_bout_path,
                dtype=numpy.dtype(np_dtype_str),
                offset=0,
                shape=(3, nx, ny, nz),
                order='C')


nlfff_data.shape

(3, 100, 296, 152)


# 获取样本原始头文件信息

sample_raw_info=raw_info[ raw_info["harpnum_trec"]==sample_harpnum_trec ]


sample_raw_info


# 获取某个样本的标签

# sample_harpnum_trec="997.20111030_142400_TAI"
sample_label=label_info[label_info["harpnum_trec"]==sample_harpnum_trec]


# 这个样本的所有标签信息
sample_label


# 获取某个有耀斑的正样本样本

sample_harpnum_trec_pos="997.20111030_142400_TAI"
sample_label_pos=label_info[label_info["harpnum_trec"]==sample_harpnum_trec_pos]


# 这个样本的所有标签信息
sample_label_pos


# 进一步也可以获取信息的耀斑信息

# 以上面样本12小时内发生耀斑信息为例
# 获取上面样本12小时内发生耀斑的id
sample_h12_flare_id=int(sample_label_pos["h12_flare_id"])


sample_h12_flare_id

67197


# 对应耀斑详细信息
sample_h12_flare_detail_info=flare_info[flare_info["deeps_flare_id"]==sample_h12_flare_id]


sample_h12_flare_detail_info


# 安装最新工具包
!pip uninstall pynlfff -y
!pip install pynlfff

# 导入工具包
import pynlfff

Found existing installation: pynlfff 0.3.3.4
Uninstalling pynlfff-0.3.3.4:
  Successfully uninstalled pynlfff-0.3.3.4
Collecting pynlfff
  Downloading pynlfff-0.3.3.5-py3-none-any.whl (44 kB)
     |████████████████████████████████| 44 kB 496 kB/s eta 0:00:01
Requirement already satisfied: h5py in /home/zander/anaconda3/lib/python3.9/site-packages (from pynlfff) (3.3.0)
Requirement already satisfied: numpy in /home/zander/anaconda3/lib/python3.9/site-packages (from pynlfff) (1.20.3)
Requirement already satisfied: matplotlib in /home/zander/anaconda3/lib/python3.9/site-packages (from pynlfff) (3.4.3)
Requirement already satisfied: kiwisolver>=1.0.1 in /home/zander/anaconda3/lib/python3.9/site-packages (from matplotlib->pynlfff) (1.3.1)
Requirement already satisfied: python-dateutil>=2.7 in /home/zander/anaconda3/lib/python3.9/site-packages (from matplotlib->pynlfff) (2.8.2)
Requirement already satisfied: pillow>=6.2.0 in /home/zander/anaconda3/lib/python3.9/site-packages (from matplotlib->pynlfff) (8.4.0)
Requirement already satisfied: cycler>=0.10 in /home/zander/anaconda3/lib/python3.9/site-packages (from matplotlib->pynlfff) (0.10.0)
Requirement already satisfied: pyparsing>=2.2.1 in /home/zander/anaconda3/lib/python3.9/site-packages (from matplotlib->pynlfff) (3.0.4)
Requirement already satisfied: six in /home/zander/anaconda3/lib/python3.9/site-packages (from cycler>=0.10->matplotlib->pynlfff) (1.16.0)
Installing collected packages: pynlfff
Successfully installed pynlfff-0.3.3.5


# 因为整个工具包依赖比较多，所以没有设置在下载工具包时全部安装

# 可以使用下面命令按照需要按照工具包
# 创建对象
checker=pynlfff.RCheck()
# 查看有哪些模块需要安装依赖
checker.h()

This module is to test some module, for pynlfff which function you need, is install or not.
    Can use RCheck().check() or RCheck().check(0) for Full module test
    RCheck().check(1) for Base function module test 
    RCheck().check(2) for Prepare function module test 
    RCheck().check(3) for Computer function module test 
    RCheck().check(4) for Product function module test 
    RCheck().check(5) for Plot function module test 
    RCheck().check(6) for Download fits raw data function module test 
    RCheck().check(7) for Label function module test


# 使用4安装和Product有关的依赖
checker.check(4)

# 使用0即按照全部依赖
checker.check(0)

Start Test
<module 'h5py' from '/home/zander/anaconda3/lib/python3.9/site-packages/h5py/__init__.py'>
True
<module 'numpy' from '/home/zander/anaconda3/lib/python3.9/site-packages/numpy/__init__.py'>
True
Test Finish, Result: [['h5py', 'Exists'], ['numpy', 'Exists']]
['h5py', 'Exists']
['numpy', 'Exists']
Start Test
<module 'pandas' from '/home/zander/anaconda3/lib/python3.9/site-packages/pandas/__init__.py'>
True
<module 'astropy' from '/home/zander/anaconda3/lib/python3.9/site-packages/astropy/__init__.py'>
True
<module 'numpy' from '/home/zander/anaconda3/lib/python3.9/site-packages/numpy/__init__.py'>
True
<module 'sunpy' from '/home/zander/anaconda3/lib/python3.9/site-packages/sunpy/__init__.py'>
True
<module 'drms' from '/home/zander/anaconda3/lib/python3.9/site-packages/drms/__init__.py'>
True
<module 'h5py' from '/home/zander/anaconda3/lib/python3.9/site-packages/h5py/__init__.py'>
True
<module 'matplotlib' from '/home/zander/anaconda3/lib/python3.9/site-packages/matplotlib/__init__.py'>
True
<module 'psutil' from '/home/zander/anaconda3/lib/python3.9/site-packages/psutil/__init__.py'>
True
<module 'requests' from '/home/zander/anaconda3/lib/python3.9/site-packages/requests/__init__.py'>
True
<module 'bs4' from '/home/zander/anaconda3/lib/python3.9/site-packages/bs4/__init__.py'>
True
<module 'lxml' from '/home/zander/anaconda3/lib/python3.9/site-packages/lxml/__init__.py'>
True
<module 'zeep' from '/home/zander/anaconda3/lib/python3.9/site-packages/zeep/__init__.py'>
True
<module 'wget' from '/home/zander/anaconda3/lib/python3.9/site-packages/wget.py'>
True
<module 'peewee' from '/home/zander/anaconda3/lib/python3.9/site-packages/peewee.py'>
True
<module 'vtk' from '/home/zander/anaconda3/lib/python3.9/site-packages/vtk.py'>
True
Test Finish, Result: [['pandas', 'Exists'], ['astropy', 'Exists'], ['numpy', 'Exists'], ['sunpy', 'Exists'], ['drms', 'Exists'], ['h5py', 'Exists'], ['matplotlib', 'Exists'], ['psutil', 'Exists'], ['requests', 'Exists'], ['bs4', 'Exists'], ['lxml', 'Exists'], ['zeep', 'Exists'], ['wget', 'Exists'], ['peewee', 'Exists'], ['vtk', 'Exists']]
['pandas', 'Exists']
['astropy', 'Exists']
['numpy', 'Exists']
['sunpy', 'Exists']
['drms', 'Exists']
['h5py', 'Exists']
['matplotlib', 'Exists']
['psutil', 'Exists']
['requests', 'Exists']
['bs4', 'Exists']
['lxml', 'Exists']
['zeep', 'Exists']
['wget', 'Exists']
['peewee', 'Exists']
['vtk', 'Exists']

[['pandas', 'Exists'],
 ['astropy', 'Exists'],
 ['numpy', 'Exists'],
 ['sunpy', 'Exists'],
 ['drms', 'Exists'],
 ['h5py', 'Exists'],
 ['matplotlib', 'Exists'],
 ['psutil', 'Exists'],
 ['requests', 'Exists'],
 ['bs4', 'Exists'],
 ['lxml', 'Exists'],
 ['zeep', 'Exists'],
 ['wget', 'Exists'],
 ['peewee', 'Exists'],
 ['vtk', 'Exists']]


# 在数据集摘要信息里面有大于或等于一个样本，
# 我们以这个样本为例进行说明

# 以0号样本为例，读取已经下载样本的`harpnum_trec`和路径
nlfff_path_dataset_summary="nlfff-dataset-info"
sample_num=0
sample_harpnum_trec=config_ds["demo_sample"][sample_num]["harpnum_trec"]
sample_path=os.path.join(
    nlfff_path_dataset_summary,
    config_ds["demo_sample"][sample_num]["path"])


# 样本的harp number 和时间
sample_harpnum_trec

'7327.20181220_093600_TAI'


# Bout path
sample_bout_path=os.path.join(sample_path,"Bout.bin")
sample_bout_path

'nlfff-dataset-info/demo/hmi.sharp_cea_720s.7327.20181220_093600_TAI/Bout.bin'


# 获取样本产品文件信息

sample_product_info=product_info[ product_info["harpnum_trec"]==sample_harpnum_trec ]
sample_product_info


# 样本的最大计算等级 
sample_bout_maxlevel=int(sample_product_info["bout_maxlevel"])
# 样本的nx，ny，nz分别为
sample_nx=int(sample_product_info["grid_x"])
sample_ny=int(sample_product_info["grid_y"])
sample_nz=int(sample_product_info["grid_z"])
# 样本的identifiers为
sample_identifiers=int(sample_product_info["identifiers"])


from pynlfff.pyproduct import file

# 创建读取对象
r=file.NlfffFile()


r = file.NlfffFile()


bout_data = r.read_bin(sample_bout_path, nx=sample_nx,ny=sample_ny,nz=sample_nz)
print(bout_data.shape)
# print(s)

(3, 100, 296, 152)


bout_data

memmap([[[[-1.31044950e+01, -1.07726199e+01, -9.31814325e+00, ...,
           -6.59102380e-01, -6.59102380e-01, -6.59102380e-01],
          [-1.51931409e+01, -1.22127697e+01, -9.59159267e+00, ...,
           -6.62805868e-01, -6.62805868e-01, -6.62805868e-01],
          [-2.27258283e+01, -1.36529194e+01, -9.86504208e+00, ...,
           -6.66509356e-01, -6.66509356e-01, -6.66509356e-01],
          ...,
          [ 1.58692752e+01,  3.32809154e+00,  2.23416430e+00, ...,
            2.44938261e-01,  2.44938261e-01,  2.44938261e-01],
          [ 9.62912943e+00,  2.83513561e+00,  2.23416430e+00, ...,
            2.44938261e-01,  2.44938261e-01,  2.44938261e-01],
          [ 5.19271162e+00,  2.83513561e+00,  2.23416430e+00, ...,
            2.44938261e-01,  2.44938261e-01,  2.44938261e-01]],

         [[-1.51724241e+01, -1.02950914e+01, -8.20474653e+00, ...,
           -6.52507450e-01, -6.52507450e-01, -6.52507450e-01],
          [-1.44053712e+01, -1.14289234e+01, -8.58725953e+00, ...,
           -6.60360707e-01, -6.57826568e-01, -6.56187112e-01],
          [-1.68870379e+01, -1.42346166e+01, -9.05483154e+00, ...,
           -6.64460732e-01, -6.61686425e-01, -6.59866773e-01],
          ...,
          [-3.54836018e+00,  1.46486296e+00,  5.72287187e-01, ...,
            2.37319136e-01,  2.36572991e-01,  2.35982653e-01],
          [-8.85968063e+00,  1.46227195e+00,  6.43827933e-01, ...,
            2.37350424e-01,  2.36506795e-01,  2.35982653e-01],
          [-6.50229845e+00, -1.67490716e+00, -4.54792534e-02, ...,
            2.35982653e-01,  2.35982653e-01,  2.35982653e-01]],

         [[-1.62226139e+01, -9.81756298e+00, -7.09134980e+00, ...,
           -6.45912520e-01, -6.45912520e-01, -6.45912520e-01],
          [-1.38035766e+01, -9.71475911e+00, -7.22248697e+00, ...,
           -6.54930419e-01, -6.51774346e-01, -6.49568355e-01],
          [-1.48065590e+01, -1.24087737e+01, -8.23603428e+00, ...,
           -6.59895328e-01, -6.56056616e-01, -6.53224190e-01],
          ...,
          [-1.36240982e+01, -7.44816370e+00, -3.85904540e+00, ...,
            2.31412517e-01,  2.29372410e-01,  2.27027045e-01],
          [-1.27016182e+01, -8.27758909e+00, -3.86570880e+00, ...,
            2.30730356e-01,  2.28920863e-01,  2.27027045e-01],
          [-9.61784709e+00, -6.18494992e+00, -2.32512281e+00, ...,
            2.27027045e-01,  2.27027045e-01,  2.27027045e-01]],

         ...,

         [[ 2.65698202e+01,  2.61640129e+01,  2.47086401e+01, ...,
            3.48661199e-01,  3.48661199e-01,  3.48661199e-01],
          [ 2.31269413e+01,  2.50436836e+01,  2.44801886e+01, ...,
            3.43999870e-01,  3.45554837e-01,  3.50740827e-01],
          [ 1.77614886e+01,  2.12224817e+01,  2.32910109e+01, ...,
            3.46384715e-01,  3.48258325e-01,  3.52820455e-01],
          ...,
          [-2.73636923e-01, -2.95356138e+00, -7.21513480e+00, ...,
           -5.75435624e-01, -5.74366052e-01, -5.73294373e-01],
          [ 6.68685970e-01, -4.97443986e+00, -8.15128005e+00, ...,
           -5.73772512e-01, -5.73215432e-01, -5.73294373e-01],
          [-1.36614229e+00, -6.29618506e+00, -7.54300828e+00, ...,
           -5.73294373e-01, -5.73294373e-01, -5.73294373e-01]],

         [[ 2.23678771e+01,  2.39879184e+01,  2.47086401e+01, ...,
            3.48661199e-01,  3.48661199e-01,  3.48661199e-01],
          [ 1.80660345e+01,  2.25698427e+01,  2.34568643e+01, ...,
            3.49268518e-01,  3.49212804e-01,  3.50740827e-01],
          [ 1.42964605e+01,  1.92277744e+01,  2.21292769e+01, ...,
            3.51401893e-01,  3.51455124e-01,  3.52820455e-01],
          ...,
          [-3.70921716e+00, -4.50725090e+00, -7.41031154e+00, ...,
           -5.74330316e-01, -5.73708403e-01, -5.73294373e-01],
          [ 5.50775292e+00, -2.39147457e+00, -6.51989304e+00, ...,
           -5.73538680e-01, -5.73158879e-01, -5.73294373e-01],
          [ 2.25973450e+00, -4.69063304e+00, -7.54300828e+00, ...,
           -5.73294373e-01, -5.73294373e-01, -5.73294373e-01]],

         [[ 1.76555365e+01,  2.39879184e+01,  2.47086401e+01, ...,
            3.48661199e-01,  3.48661199e-01,  3.48661199e-01],
          [ 1.69437098e+01,  2.13044874e+01,  2.34641682e+01, ...,
            3.50740827e-01,  3.50740827e-01,  3.50740827e-01],
          [ 1.66954722e+01,  1.86210564e+01,  2.22196962e+01, ...,
            3.52820455e-01,  3.52820455e-01,  3.52820455e-01],
          ...,
          [-4.91528467e+00, -5.82091821e+00, -7.54300828e+00, ...,
           -5.73294373e-01, -5.73294373e-01, -5.73294373e-01],
          [ 4.42433594e+00, -4.69063304e+00, -7.54300828e+00, ...,
           -5.73294373e-01, -5.73294373e-01, -5.73294373e-01],
          [ 2.68535675e+00, -4.69063304e+00, -7.54300828e+00, ...,
           -5.73294373e-01, -5.73294373e-01, -5.73294373e-01]]],


        [[[-2.67670497e+01, -2.25152001e+01, -2.04868811e+01, ...,
           -2.08801030e-01, -2.08801030e-01, -2.08801030e-01],
          [-2.51385478e+01, -2.18916655e+01, -1.83235556e+01, ...,
           -1.97911847e-01, -1.97911847e-01, -1.97911847e-01],
          [-2.85658013e+01, -2.12681308e+01, -1.61602301e+01, ...,
           -1.87022664e-01, -1.87022664e-01, -1.87022664e-01],
          ...,
          [-2.95981882e+01, -1.72691696e+01, -1.21571192e+01, ...,
           -1.02677648e-01, -1.02677648e-01, -1.02677648e-01],
          [-1.71524835e+01, -1.84816307e+01, -1.21571192e+01, ...,
           -1.02677648e-01, -1.02677648e-01, -1.02677648e-01],
          [-2.04185727e+01, -1.84816307e+01, -1.21571192e+01, ...,
           -1.02677648e-01, -1.02677648e-01, -1.02677648e-01]],

         [[-3.11160823e+01, -2.48073797e+01, -2.07034690e+01, ...,
           -2.12516097e-01, -2.12516097e-01, -2.12516097e-01],
          [-2.84270985e+01, -2.40571643e+01, -2.00794420e+01, ...,
           -2.05379378e-01, -2.03295443e-01, -2.01597298e-01],
          [-3.00767080e+01, -2.28881370e+01, -1.93103421e+01, ...,
           -1.95925588e-01, -1.93365675e-01, -1.90678499e-01],
          ...,
          [-2.30762165e+01, -2.04364817e+01, -1.41811789e+01, ...,
           -1.00283153e-01, -1.00380659e-01, -1.03781461e-01],
          [-2.26682784e+01, -2.19042827e+01, -1.37359878e+01, ...,
           -1.03683230e-01, -1.02989762e-01, -1.03781461e-01],
          [-2.94124834e+01, -2.19047512e+01, -1.19250553e+01, ...,
           -1.03781461e-01, -1.03781461e-01, -1.03781461e-01]],

         [[-2.88794802e+01, -2.70995592e+01, -2.09200568e+01, ...,
           -2.16231163e-01, -2.16231163e-01, -2.16231163e-01],
          [-3.01440877e+01, -2.60383186e+01, -2.09649163e+01, ...,
           -2.09334155e-01, -2.06914247e-01, -2.05282749e-01],
          [-3.53707639e+01, -2.63834378e+01, -2.13554601e+01, ...,
           -2.01248779e-01, -1.97478598e-01, -1.94334334e-01],
          ...,
          [-2.79820620e+01, -2.04184069e+01, -1.48595173e+01, ...,
           -1.01777506e-01, -1.01965952e-01, -1.04885274e-01],
          [-4.10431063e+01, -2.49123885e+01, -1.57314997e+01, ...,
           -1.04902272e-01, -1.04234869e-01, -1.04885274e-01],
          [-4.04297586e+01, -2.53278717e+01, -1.16929913e+01, ...,
           -1.04885274e-01, -1.04885274e-01, -1.04885274e-01]],

         ...,

         [[-1.62683325e+01, -1.62658801e+01, -1.48409055e+01, ...,
           -3.18822642e-01, -3.18822642e-01, -3.18822642e-01],
          [-1.62570995e+01, -1.55577437e+01, -1.29127068e+01, ...,
           -3.11377319e-01, -3.09338787e-01, -3.07499417e-01],
          [-2.17673428e+01, -1.77323418e+01, -1.28986179e+01, ...,
           -3.05632632e-01, -3.00944019e-01, -2.96176192e-01],
          ...,
          [-2.76910141e+01, -1.74524268e+01, -1.37194241e+01, ...,
            4.04452301e-02,  3.90627822e-02,  3.52440062e-02],
          [-1.93527245e+01, -1.72753294e+01, -1.27122101e+01, ...,
            3.74224606e-02,  3.67205232e-02,  3.52440062e-02],
          [-5.81451216e+00, -1.52678896e+01, -1.02138230e+01, ...,
            3.52440062e-02,  3.52440062e-02,  3.52440062e-02]],

         [[-2.10050470e+01, -1.77386267e+01, -1.48409055e+01, ...,
           -3.18822642e-01, -3.18822642e-01, -3.18822642e-01],
          [-2.18759322e+01, -1.66845986e+01, -1.36050946e+01, ...,
           -3.10826041e-01, -3.08995063e-01, -3.07499417e-01],
          [-1.87930035e+01, -1.60993191e+01, -1.29446996e+01, ...,
           -3.04191016e-01, -3.00402557e-01, -2.96176192e-01],
          ...,
          [-2.65976314e+01, -1.78504883e+01, -1.33124233e+01, ...,
            3.97576202e-02,  3.94595585e-02,  3.52440062e-02],
          [-1.82518194e+01, -1.52351135e+01, -1.10655539e+01, ...,
            3.72067978e-02,  3.70316129e-02,  3.52440062e-02],
          [-6.83084563e+00, -1.57585155e+01, -1.02138230e+01, ...,
            3.52440062e-02,  3.52440062e-02,  3.52440062e-02]],

         [[-1.91202026e+01, -1.77386267e+01, -1.48409055e+01, ...,
           -3.18822642e-01, -3.18822642e-01, -3.18822642e-01],
          [-2.15633270e+01, -1.72611422e+01, -1.48310588e+01, ...,
           -3.07499417e-01, -3.07499417e-01, -3.07499417e-01],
          [-2.15627099e+01, -1.67836577e+01, -1.48212122e+01, ...,
           -2.96176192e-01, -2.96176192e-01, -2.96176192e-01],
          ...,
          [-1.92399853e+01, -1.32307423e+01, -1.02138230e+01, ...,
            3.52440062e-02,  3.52440062e-02,  3.52440062e-02],
          [-1.76259994e+01, -1.57585155e+01, -1.02138230e+01, ...,
            3.52440062e-02,  3.52440062e-02,  3.52440062e-02],
          [-1.26887218e+01, -1.57585155e+01, -1.02138230e+01, ...,
            3.52440062e-02,  3.52440062e-02,  3.52440062e-02]]],


        [[[-1.25568691e+01, -3.63734765e+00,  4.26934688e-01, ...,
            1.56327637e+00,  1.56327637e+00,  1.56327637e+00],
          [-7.08903640e+00,  5.58251621e-01,  3.97459783e+00, ...,
            1.57197462e+00,  1.57197462e+00,  1.57197462e+00],
          [-1.14758675e+01,  4.75385089e+00,  7.52226097e+00, ...,
            1.58067286e+00,  1.58067286e+00,  1.58067286e+00],
          ...,
          [ 1.47498835e+01, -1.11370350e+00, -2.55596764e+00, ...,
           -1.52850141e+00, -1.52850141e+00, -1.52850141e+00],
          [ 1.78436871e+01,  3.12553805e+00, -2.55596764e+00, ...,
           -1.52850141e+00, -1.52850141e+00, -1.52850141e+00],
          [ 2.31120637e+01,  3.12553805e+00, -2.55596764e+00, ...,
           -1.52850141e+00, -1.52850141e+00, -1.52850141e+00]],

         [[-1.38043562e+01, -3.85059525e+00,  9.03193560e-01, ...,
            1.57321548e+00,  1.57321548e+00,  1.57321548e+00],
          [-3.23470714e+00, -4.16018212e-01,  1.94594457e+00, ...,
            1.58479641e+00,  1.58286120e+00,  1.58201088e+00],
          [-5.65077822e-01,  3.30084924e+00,  5.71149574e+00, ...,
            1.59546865e+00,  1.59253304e+00,  1.59080628e+00],
          ...,
          [-1.17262496e+01, -3.64092337e+00, -4.63374478e+00, ...,
           -1.54373267e+00, -1.53868381e+00, -1.53385555e+00],
          [ 1.50436700e+00,  7.88626084e-01, -1.37803351e+00, ...,
           -1.53964635e+00, -1.53621682e+00, -1.53385555e+00],
          [ 1.58384235e+01,  2.11202417e+00, -1.99169425e+00, ...,
           -1.53385555e+00, -1.53385555e+00, -1.53385555e+00]],

         [[-1.36876646e+01, -4.06384286e+00,  1.37945243e+00, ...,
            1.58315458e+00,  1.58315458e+00,  1.58315458e+00],
          [-6.94444318e+00, -1.34377169e+00,  1.59705613e+00, ...,
            1.59651792e+00,  1.59373550e+00,  1.59204714e+00],
          [-7.15333851e-01,  2.01921434e+00,  3.37710102e+00, ...,
            1.60824144e+00,  1.60404185e+00,  1.60093970e+00],
          ...,
          [-4.20760393e+01, -2.02449911e+01, -1.11003431e+01, ...,
           -1.55366119e+00, -1.54640858e+00, -1.53920968e+00],
          [-1.47993216e+01, -8.91304327e+00, -5.04027288e+00, ...,
           -1.54751576e+00, -1.54280594e+00, -1.53920968e+00],
          [ 1.23213648e+01,  1.09851030e+00, -1.42742086e+00, ...,
           -1.53920968e+00, -1.53920968e+00, -1.53920968e+00]],

         ...,

         [[-9.81503980e+00, -8.90668518e+00, -4.63403331e+00, ...,
            1.81371842e+00,  1.81371842e+00,  1.81371842e+00],
          [-2.57478506e+01, -1.04551020e+01, -3.78519063e+00, ...,
            1.83476476e+00,  1.82946347e+00,  1.82460337e+00],
          [-2.78454464e+01, -1.48304583e+01, -7.98367816e+00, ...,
            1.84959822e+00,  1.84228627e+00,  1.83548832e+00],
          ...,
          [ 3.26097491e+01,  2.20917464e+01,  1.32505260e+01, ...,
           -1.28113336e+00, -1.27370096e+00, -1.26538114e+00],
          [ 4.05441763e+01,  1.96914544e+01,  1.05568383e+01, ...,
           -1.27532480e+00, -1.27040054e+00, -1.26538114e+00],
          [ 2.83989139e+01,  1.61975399e+01,  7.40820167e+00, ...,
           -1.26538114e+00, -1.26538114e+00, -1.26538114e+00]],

         [[-1.76107734e+01, -1.10968855e+01, -4.63403331e+00, ...,
            1.81371842e+00,  1.81371842e+00,  1.81371842e+00],
          [-2.27036649e+01, -1.24252386e+01, -6.11022404e+00, ...,
            1.83050046e+00,  1.82696463e+00,  1.82460337e+00],
          [-2.32000887e+01, -1.48860816e+01, -7.72424970e+00, ...,
            1.84347711e+00,  1.83878906e+00,  1.83548832e+00],
          ...,
          [ 2.31377254e+01,  1.52988707e+01,  9.21385926e+00, ...,
           -1.27513300e+00, -1.27020665e+00, -1.26538114e+00],
          [ 3.99142344e+01,  1.99417678e+01,  1.14976231e+01, ...,
           -1.27148261e+00, -1.26809700e+00, -1.26538114e+00],
          [ 2.90517073e+01,  1.52812838e+01,  7.40820167e+00, ...,
           -1.26538114e+00, -1.26538114e+00, -1.26538114e+00]],

         [[-2.06190927e+01, -1.10968855e+01, -4.63403331e+00, ...,
            1.81371842e+00,  1.81371842e+00,  1.81371842e+00],
          [-2.19604929e+01, -1.11007737e+01, -4.50158876e+00, ...,
            1.82460337e+00,  1.82460337e+00,  1.82460337e+00],
          [-1.82218546e+01, -1.11046619e+01, -4.36914422e+00, ...,
            1.83548832e+00,  1.83548832e+00,  1.83548832e+00],
          ...,
          [ 2.14579532e+01,  1.22652075e+01,  7.40820167e+00, ...,
           -1.26538114e+00, -1.26538114e+00, -1.26538114e+00],
          [ 2.39156764e+01,  1.52812838e+01,  7.40820167e+00, ...,
           -1.26538114e+00, -1.26538114e+00, -1.26538114e+00],
          [ 1.80771953e+01,  1.52812838e+01,  7.40820167e+00, ...,
           -1.26538114e+00, -1.26538114e+00, -1.26538114e+00]]]])


r = file.NlfffFile()

h5_path = sample_bout_path.replace(".bin",".h5")

r.tran_bin2hdf5(sample_bout_path, h5_path, nx=sample_nx,ny=sample_ny,nz=sample_nz, overwrite=True)

True


from pynlfff.pyplot import plot3d_cut
d3_drawer = plot3d_cut.NlfffPlotD3CutCake()


# 手动设置色调范围
# 设置最大显示值
d3_drawer.colormap_set_max=40
# 设置最小显示值
d3_drawer.colormap_set_max=-30


# 自动设置色调范围
# 自动求的绘图切面中的最大和最小值
d3_drawer.colormap_auto_value=True
# 只有colormap_auto_value为True才生效，表示在自动取最值上再进行压缩，压缩比例，取值范围(0,1]，1表示不压缩
d3_drawer.colormap_auto_zip=0.8
# 表示取最值绝对值最大的值为最大值，其相反数为最小值，时得到的调色板关于0对称，注意如果最大值和最小值同为正或同为负，则不生效
d3_drawer.colormap_auto_mirror=True


# 越界值显示
# 默认设为True，越界值按照离得最近的那个最值取；为False，越界值不显示
d3_drawer.colormap_out_range_display=True
#


# 透明度
# 设置None则不透明
d3_drawer.colormap_alpha=None
# 设置整体透明度，取值范围(0,1)
d3_drawer.colormap_alpha=0.8
# 默认，auto自动透明，即绝对值越小透明程度越高
d3_drawer.colormap_alpha="auto"


# 设置切面边界
# 不绘制边界线
d3_drawer.cut_line_edges = None
# 默认，设置边界线样式
d3_drawer.cut_line_edges = dict(color='0.6', linewidth=0.6, zorder=1e3)


# 设置切面交线
# 不绘制切面交叉线
d3_drawer.cut_line_cross = None
# 默认，设置切面交叉线样式
d3_drawer.cut_line_cross = dict(color='0.4', linewidth=1, zorder=1e3)


# 配置数据
use_facker_data=False
if use_facker_data:
    import numpy as np
    array_data = np.random.uniform(-5, 5, size=(3, 100,50,40))
else:
    array_data=bout_data.copy()


# 加载数据
d3_drawer.load_data_array(array_data)

True


d3_drawer.add_cut(B="Bx", N="Nx", cut_num=None, cut_percent=0.5)
d3_drawer.add_cut("Bz", "Nz")
d3_drawer.add_cut("Bz", "Nz", cut_num=0)
d3_drawer.add_cut("Bz", "Nx",cut_percent=0.3)
d3_drawer.add_cut("Bz", "Nz", cut_percent=0.2)
# add cut surface for cut draw, which can be use many times for different surfance in one picture
# :param B: 'Bx' 'By' or 'Bz'; choose one Magnetic field component to draw
# :param N: 'Nx' 'Ny' or 'Nz'; choose cut in which angle; when choose 'Nz', The slice will be perpendicular to z axis
# :param cut_num: int type , set for cut where, can be [0, max(axis)], if not set this,will use cut_percent default
# :param cut_percent: default 0.5, percent of axis


# 配置信息

# 画布设置 https://matplotlib.org/stable/api/_as_gen/matplotlib.pyplot.figure.html
fig_dict=dict(figsize=[6, 6])

# 保存图片设置 https://matplotlib.org/stable/api/_as_gen/matplotlib.pyplot.savefig.html
is_save=False,
save_dict=dict(fname="./cut.png", dpi='figure')

# 展示设置 https://matplotlib.org/stable/api/_as_gen/matplotlib.pyplot.show.html
is_show=True,
show_dict=dict(block=True)

# 视角设置
view_dict=dict(
    elev=45,
    azim=45,
    dist=11
)

d3_drawer.run_cut(
    fig_dict=fig_dict,
    is_show=is_show,
    view_dict=view_dict,
    show_dict=show_dict,
    is_save=is_save,
    save_dict=save_dict
)

The Kernel crashed while executing code in the the current cell or a previous cell. Please review the code in the cell(s) to identify a possible cause of the failure. Click <a href='https://aka.ms/vscodeJupyterKernelCrash'>here</a> for more info. View Jupyter <a href='command:jupyter.viewOutput'>log</a> for further details.


 # 保存目录
out_dir="./" 

# 系统是否带gui
is_with_gui=True # 系统带gui
is_with_gui=False # 系统不带gui 如colab


# 配置数据
use_facker_data=True
if use_facker_data:
    import numpy as np
    array_data = np.random.uniform(-5, 5, size=(3, 100,50,40))
else:
    array_data=bout_data.copy()


if is_with_gui:
  from pynlfff.pyplot import plot3d_body
  import numpy as np
  
  try:
    dataBout=array_data
    result=plot3d_body.main_draw_3d_body(
        dataBout,
        out_dir,
        axis_name=['bx','by','bz']
    )
  except BaseException as e:
    print(e)
else:
  print("VTK 绘制依赖gui")

VTK 绘制依赖gui


# sample identifiers可以从产品信息表里面获得

sample_identifiers=7327201812200936003
# 其中 7327(harp number)20181220(date)093600(time)3(Max grid level).
sample_url="https://identifiers.org/nlfff:{}".format(sample_identifiers)
print("访问 {} 查看样本详细信息及下载".format(sample_url))

访问 https://identifiers.org/nlfff:7327201812200936003 查看样本详细信息及下载

	harpnum_trec	HARP_NUM	T_REC_DATETIME	T_REC_STR	FILE_NAME	query	DATE	DATE_S	DATE_B	DATE__OBS	...	continuum	Bp	Bt	Br	Bp_err	Bt_err	Br_err	conf_disambig	BHARP	MHARP
0	1644.20120516_031200_TAI	1644	2012-05-16 03:12:00.000	20120516_031200_TAI	hmi.sharp_cea_720s.1644.20120516_031200_TAI	hmi.sharp_cea_720s[1644][2012.05.16_03:12:00_TAI]	2018-05-09T03:05:47Z	2012-05-20T19:22:46Z	2015-09-30T20:47:04Z	2012-05-16T03:10:24.10Z	...	/SUM90/D1042885471/S00000/continuum.fits	/SUM90/D1042885471/S00000/Bp.fits	/SUM90/D1042885471/S00000/Bt.fits	/SUM90/D1042885471/S00000/Br.fits	/SUM90/D1042885471/S00000/Bp_err.fits	/SUM90/D1042885471/S00000/Bt_err.fits	/SUM90/D1042885471/S00000/Br_err.fits	/SUM90/D1042885471/S00000/conf_disambig.fits	InvalidLink	hmi.Mharp_720s[1644][2012.05.16_03:12:00_TAI]
1	4315.20140708_013600_TAI	4315	2014-07-08 01:36:00.000	20140708_013600_TAI	hmi.sharp_cea_720s.4315.20140708_013600_TAI	hmi.sharp_cea_720s[4315][2014.07.08_01:36:00_TAI]	2014-08-13T03:40:53Z	2014-07-12T23:18:52Z	2014-07-16T08:03:00Z	2014-07-08T01:34:25.80Z	...	/SUM22/D597857266/S00000/continuum.fits	/SUM22/D597857266/S00000/Bp.fits	/SUM22/D597857266/S00000/Bt.fits	/SUM22/D597857266/S00000/Br.fits	/SUM22/D597857266/S00000/Bp_err.fits	/SUM22/D597857266/S00000/Bt_err.fits	/SUM22/D597857266/S00000/Br_err.fits	/SUM22/D597857266/S00000/conf_disambig.fits	InvalidLink	hmi.Mharp_720s[4315][2014.07.08_01:36:00_TAI]
2	4351.20140714_142400_TAI	4351	2014-07-14 14:24:00.000	20140714_142400_TAI	hmi.sharp_cea_720s.4351.20140714_142400_TAI	hmi.sharp_cea_720s[4351][2014.07.14_14:24:00_TAI]	2014-08-19T03:03:14Z	2014-07-18T23:26:26Z	2014-07-22T00:47:31Z	2014-07-14T14:22:25.70Z	...	/SUM1/D599938236/S00000/continuum.fits	/SUM1/D599938236/S00000/Bp.fits	/SUM1/D599938236/S00000/Bt.fits	/SUM1/D599938236/S00000/Br.fits	/SUM1/D599938236/S00000/Bp_err.fits	/SUM1/D599938236/S00000/Bt_err.fits	/SUM1/D599938236/S00000/Br_err.fits	/SUM1/D599938236/S00000/conf_disambig.fits	InvalidLink	hmi.Mharp_720s[4351][2014.07.14_14:24:00_TAI]
3	536.20110426_031200_TAI	536	2011-04-26 03:12:00.000	20110426_031200_TAI	hmi.sharp_cea_720s.536.20110426_031200_TAI	hmi.sharp_cea_720s[536][2011.04.26_03:12:00_TAI]	2018-04-29T14:50:56Z	2012-09-05T09:08:03Z	2015-05-12T15:15:34Z	2011-04-26T03:10:21.60Z	...	/SUM90/D1038869818/S00000/continuum.fits	/SUM90/D1038869818/S00000/Bp.fits	/SUM90/D1038869818/S00000/Bt.fits	/SUM90/D1038869818/S00000/Br.fits	/SUM90/D1038869818/S00000/Bp_err.fits	/SUM90/D1038869818/S00000/Bt_err.fits	/SUM90/D1038869818/S00000/Br_err.fits	/SUM90/D1038869818/S00000/conf_disambig.fits	InvalidLink	hmi.Mharp_720s[536][2011.04.26_03:12:00_TAI]
4	5415.20150409_080000_TAI	5415	2015-04-09 08:00:00.000	20150409_080000_TAI	hmi.sharp_cea_720s.5415.20150409_080000_TAI	hmi.sharp_cea_720s[5415][2015.04.09_08:00:00_TAI]	2015-05-15T03:15:44Z	2015-04-13T14:07:11Z	2015-04-14T18:23:45Z	2015-04-09T07:58:18.30Z	...	/SUM35/D698134494/S00000/continuum.fits	/SUM35/D698134494/S00000/Bp.fits	/SUM35/D698134494/S00000/Bt.fits	/SUM35/D698134494/S00000/Br.fits	/SUM35/D698134494/S00000/Bp_err.fits	/SUM35/D698134494/S00000/Bt_err.fits	/SUM35/D698134494/S00000/Br_err.fits	/SUM35/D698134494/S00000/conf_disambig.fits	InvalidLink	hmi.Mharp_720s[5415][2015.04.09_08:00:00_TAI]
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
73742	6258.20160115_222400_TAI	6258	2016-01-15 22:24:00.000	20160115_222400_TAI	hmi.sharp_cea_720s.6258.20160115_222400_TAI	hmi.sharp_cea_720s[6258][2016.01.15_22:24:00_TAI]	2016-02-20T04:08:10Z	2016-01-19T17:42:19Z	2016-01-20T19:17:24Z	2016-01-15T22:22:08.30Z	...	/SUM80/D789029067/S00000/continuum.fits	/SUM80/D789029067/S00000/Bp.fits	/SUM80/D789029067/S00000/Bt.fits	/SUM80/D789029067/S00000/Br.fits	/SUM80/D789029067/S00000/Bp_err.fits	/SUM80/D789029067/S00000/Bt_err.fits	/SUM80/D789029067/S00000/Br_err.fits	/SUM80/D789029067/S00000/conf_disambig.fits	InvalidLink	hmi.Mharp_720s[6258][2016.01.15_22:24:00_TAI]
73743	6599.20160607_080000_TAI	6599	2016-06-07 08:00:00.000	20160607_080000_TAI	hmi.sharp_cea_720s.6599.20160607_080000_TAI	hmi.sharp_cea_720s[6599][2016.06.07_08:00:00_TAI]	2017-12-05T18:45:13Z	2017-11-04T07:17:18Z	2017-11-26T22:59:24Z	2016-06-07T07:58:46.60Z	...	/SUM95/D996809863/S00000/continuum.fits	/SUM95/D996809863/S00000/Bp.fits	/SUM95/D996809863/S00000/Bt.fits	/SUM95/D996809863/S00000/Br.fits	/SUM95/D996809863/S00000/Bp_err.fits	/SUM95/D996809863/S00000/Bt_err.fits	/SUM95/D996809863/S00000/Br_err.fits	/SUM95/D996809863/S00000/conf_disambig.fits	InvalidLink	hmi.Mharp_720s[6599][2016.06.07_08:00:00_TAI]
73744	7255.20180512_080000_TAI	7255	2018-05-12 08:00:00.000	20180512_080000_TAI	hmi.sharp_cea_720s.7255.20180512_080000_TAI	hmi.sharp_cea_720s[7255][2018.05.12_08:00:00_TAI]	2018-06-17T03:05:37Z	2018-05-16T20:22:23Z	2018-05-18T07:52:28Z	2018-05-12T07:58:43.20Z	...	/SUM80/D1055843035/S00000/continuum.fits	/SUM80/D1055843035/S00000/Bp.fits	/SUM80/D1055843035/S00000/Bt.fits	/SUM80/D1055843035/S00000/Br.fits	/SUM80/D1055843035/S00000/Bp_err.fits	/SUM80/D1055843035/S00000/Bt_err.fits	/SUM80/D1055843035/S00000/Br_err.fits	/SUM80/D1055843035/S00000/conf_disambig.fits	InvalidLink	hmi.Mharp_720s[7255][2018.05.12_08:00:00_TAI]
73745	892.20110926_080000_TAI	892	2011-09-26 08:00:00.000	20110926_080000_TAI	hmi.sharp_cea_720s.892.20110926_080000_TAI	hmi.sharp_cea_720s[892][2011.09.26_08:00:00_TAI]	2018-04-28T01:47:53Z	2011-09-30T18:34:03Z	2015-07-02T20:56:15Z	2011-09-26T07:58:20.00Z	...	/SUM87/D1038232342/S00000/continuum.fits	/SUM87/D1038232342/S00000/Bp.fits	/SUM87/D1038232342/S00000/Bt.fits	/SUM87/D1038232342/S00000/Br.fits	/SUM87/D1038232342/S00000/Bp_err.fits	/SUM87/D1038232342/S00000/Bt_err.fits	/SUM87/D1038232342/S00000/Br_err.fits	/SUM87/D1038232342/S00000/conf_disambig.fits	InvalidLink	hmi.Mharp_720s[892][2011.09.26_08:00:00_TAI]
73746	1422.20120301_160000_TAI	1422	2012-03-01 16:00:00.000	20120301_160000_TAI	hmi.sharp_cea_720s.1422.20120301_160000_TAI	hmi.sharp_cea_720s[1422][2012.03.01_16:00:00_TAI]	2018-04-29T18:00:12Z	2012-09-05T09:13:59Z	2015-09-01T05:38:35Z	2012-03-01T15:58:13.90Z	...	/SUM94/D1038926095/S00000/continuum.fits	/SUM94/D1038926095/S00000/Bp.fits	/SUM94/D1038926095/S00000/Bt.fits	/SUM94/D1038926095/S00000/Br.fits	/SUM94/D1038926095/S00000/Bp_err.fits	/SUM94/D1038926095/S00000/Bt_err.fits	/SUM94/D1038926095/S00000/Br_err.fits	/SUM94/D1038926095/S00000/conf_disambig.fits	InvalidLink	hmi.Mharp_720s[1422][2012.03.01_16:00:00_TAI]

	harpnum_trec	sharp_num	t_rec	bout_path	bout_md5	bout_size	grid_x	grid_y	grid_z	grid_xyz	bout_maxlevel	bout_quality	batch	notes	bout_quality_value	identifiers
0	6558.20160520_044800_TAI	6558	2016-05-20 04:48:00.000	grid3.done/num_6500_6599/hmi.sharp_cea_720s.65...	ea4c960822a1284fe43637b1717bc30b	3580360704	432	764	452	149181696	3	True	20220301	new	15.568000	6558201605200448003
1	6558.20160520_044800_TAI	6558	2016-05-20 04:48:00.000	grid2.done/hmi.sharp_cea_720s.6558.20160520_04...	bf41326b2b90597a539a796c7eb72e10	447545088	216	382	226	18647712	2	True	20220301	old	16.364700	6558201605200448002
2	2887.20130629_111200_TAI	2887	2013-06-29 11:12:00.000	grid2.done/hmi.sharp_cea_720s.2887.20130629_11...	c3661aaef993bed04a10fb74483e7613	456693120	214	390	228	19028880	2	True	20220301	old	17.953400	2887201306291112002
3	2887.20130629_111200_TAI	2887	2013-06-29 11:12:00.000	grid3.done/num_2800_2899/hmi.sharp_cea_720s.28...	38f40bf4cd8e7fa66d2563570ccbbd3f	3653544960	428	780	456	152231040	3	True	20220301	new	18.468100	2887201306291112003
4	5183.20150215_080000_TAI	5183	2015-02-15 08:00:00.000	grid2.done/hmi.sharp_cea_720s.5183.20150215_08...	7de9b081558f9069f4a4732f8bdcb42f	198342144	168	286	172	8264256	2	True	20220301	old	17.937401	5183201502150800002
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
73742	4616.20141003_142400_TAI	4616	2014-10-03 14:24:00.000	grid3.done/num_4600_4699/hmi.sharp_cea_720s.46...	7c3da7a4dbd08da4ccee60fecd7e0118	1635068160	792	1012	680	545022720	3	True	20220301	need check size	17.964899	4616201410031424003
73743	4698.20141024_160000_TAI	4698	2014-10-24 16:00:00.000	grid3.done/num_4600_4699/hmi.sharp_cea_720s.46...	851980aacc23901e5ff3a654debe013f	3024076800	800	1472	856	1008025600	3	True	20220301	need check size	16.359800	4698201410241600003
73744	4640.20141006_173600_TAI	4640	2014-10-06 17:36:00.000	grid3.done/num_4600_4699/hmi.sharp_cea_720s.46...	e99415002ee27c14dcf7d08490a15840	1137168000	500	1192	636	379056000	3	True	20220301	need check size	23.332500	4640201410061736003
73745	5039.20150112_013600_TAI	5039	2015-01-12 01:36:00.000	grid3.done/num_5000_5099/hmi.sharp_cea_720s.50...	d64654bd4ed69f201c34faf0b036b9c7	5301338112	1008	1712	1024	1767112704	3	True	20220301	need check size	17.475500	5039201501120136003
73746	5028.20150109_044800_TAI	5028	2015-01-09 04:48:00.000	grid3.done/num_5000_5099/hmi.sharp_cea_720s.50...	21f9ab2c5e376aec3f45cfbe487e3e23	1806686976	656	1268	724	602228992	3	True	20220301	need check size	22.137501	5028201501090448003

	deeps_flare_id	start_datetime	end_datetime	peak_datetime	xray_class	xray_intensity	latitude	longtitude	noaa_ar	source
0	78524	2016-01-01 23:10:00.000	2016-01-02 01:01:00.000	2016-01-02 00:11:00.000	M	23.0	NaN	NaN	12473.0	swpc
1	32340	1991-12-18 06:06:00.000	1991-12-18 06:39:00.000	1991-12-18 06:38:00.000	C	19.0	NaN	NaN	NaN	goes
2	19726	1986-05-02 16:36:00.000	1986-05-02 16:44:00.000	1986-05-02 16:41:00.000	C	12.0	NaN	NaN	NaN	goes
3	36328	1993-05-26 01:54:00.000	1993-05-26 02:03:00.000	1993-05-26 01:59:00.000	C	32.0	NaN	NaN	NaN	goes
4	26657	1990-01-23 05:03:00.000	1990-01-23 05:35:00.000	1990-01-23 05:04:00.000	M	23.0	-10.0	56.0	5900.0	goes
...	...	...	...	...	...	...	...	...	...	...
85509	72167	2014-03-12 22:28:00.000	2014-03-12 22:39:00.000	2014-03-12 22:34:00.000	M	93.0	15.0	-78.0	11996.0	goes
85510	44281	1998-12-18 15:59:00.000	1998-12-18 16:13:00.000	1998-12-18 16:05:00.000	C	20.0	19.0	65.0	8415.0	goes
85511	30992	1991-07-20 06:51:00.000	1991-07-20 07:07:00.000	1991-07-20 06:54:00.000	C	34.0	-14.0	72.0	6737.0	goes
85512	76690	2016-08-09 01:55:00.000	2016-08-09 02:09:00.000	2016-08-09 01:58:00.000	B	76.0	NaN	NaN	NaN	goes
85513	11311	1982-06-05 12:39:00.000	1982-06-05 12:46:00.000	1982-06-05 12:42:00.000	C	38.0	NaN	NaN	NaN	goes

	harpnum_trec	harp_num	t_rec_str	now_flare_level	now_flare_id	h6_flare_level	h6_flare_id	h12_flare_level	h12_flare_id	h24_flare_level	h24_flare_id	h48_flare_level	h48_flare_id	t_rec_datetime	h24_posmx	h24_poscmx	h48_posmx	h48_poscmx	h24_delta05	h48_delta05
0	1.20100504_160000_TAI	1	20100504_160000_TAI	0	-1	0	-1	0	-1	0	-1	0	-1	2010-05-04 16:00:00.000	0	0	0	0	0	0
1	1.20100504_173600_TAI	1	20100504_173600_TAI	0	-1	0	-1	0	-1	0	-1	0	-1	2010-05-04 17:36:00.000	0	0	0	0	0	0
2	1.20100504_191200_TAI	1	20100504_191200_TAI	0	-1	0	-1	0	-1	0	-1	0	-1	2010-05-04 19:12:00.000	0	0	0	0	0	0
3	1.20100504_204800_TAI	1	20100504_204800_TAI	0	-1	0	-1	0	-1	0	-1	0	-1	2010-05-04 20:48:00.000	0	0	0	0	0	0
4	1.20100504_222400_TAI	1	20100504_222400_TAI	0	-1	0	-1	0	-1	0	-1	0	-1	2010-05-04 22:24:00.000	0	0	0	0	0	0
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
46109	997.20111030_080000_TAI	997	20111030_080000_TAI	0	-1	324	67189	324	67189	324	67189	324	67189	2011-10-30 08:00:00.000	0	1	0	1	3	3
46110	997.20111030_093600_TAI	997	20111030_093600_TAI	324	67189	324	67189	324	67189	324	67189	324	67189	2011-10-30 09:36:00.000	0	2	0	2	0	0
46111	997.20111030_111200_TAI	997	20111030_111200_TAI	0	-1	319	67192	319	67192	323	67197	323	67197	2011-10-30 11:12:00.000	0	1	0	1	3	3
46112	997.20111030_124800_TAI	997	20111030_124800_TAI	0	-1	319	67192	319	67192	323	67197	323	67197	2011-10-30 12:48:00.000	0	1	0	1	3	3
46113	997.20111030_142400_TAI	997	20111030_142400_TAI	0	-1	319	67192	323	67197	323	67197	323	67197	2011-10-30 14:24:00.000	0	1	0	1	3	3

写在前面¶

说明¶

数据集建造¶

三维磁场数据产品使用¶

数据集和标签¶

下载数据集摘要信息¶

下载标签信息¶

下载数据库归档信息¶

使用数据集摘要信息¶

读取配置¶

读取原始头文件信息¶

读取产品文件信息¶

使用标签信息¶

读取配置¶

标签信息¶

耀斑信息¶

使用数据库归档信息¶

读取配置¶

归档文件的基本介绍¶

基本使用¶

查询表数据¶

高级查询¶

使用单个样本（基本使用）¶

基本读取¶

获取存储路径和时间¶

获取样本产品文件信息（nx ny nz）¶

读取数据¶

获取关联信息¶

获取样本原始头文件信息¶

获取标签信息¶

获取二级关联信息¶

标签对应耀斑信息¶

使用单个样本（pynlfff工具包）¶

工具包初始化¶

安装pynlfff工具包¶

安装工具包依赖¶

数据读取¶

获取存储路径和时间¶

获取样本产品文件信息（nx ny nz）¶

读取数据¶

格式转换¶

转HDF¶

可视化¶

三维切割绘制¶

创建绘图对象¶

设置绘图对象属性（可选）¶

加载数据¶

添加绘图切面¶

执行绘图操作¶

三维体绘制¶

更多样本获取方法¶

方法1 通过官方网站查询下载¶

方法2 通过identifiers跳转在线查看样本¶

方法3 在线申请邮寄硬盘拷贝¶