View Issue Details

IDProjectCategoryView StatusLast Update
0005541SUMoBugpublic2019-06-10 23:20
ReporterwolfAssigned ToKyle_Katarn 
PrioritynormalSeverityminorReproducibilityalways
Status acknowledgedResolutionopen 
Platformi7-8000 notebookOSWindows 10OS Version1809 64-bit
Product Version5.9.3 
Target VersionShort termFixed in Version 
Summary0005541: scan action takes very long without any display changes
DescriptionAs blocking action is still not documented and not yet further detailed of the related issues, I may report that on my notebook with sufficient memory and power, the scan action takes 20 minutes, the check action 6-9 minutes. I configured scan action not to do additional load and check actions. I further configured it to to deep scanning, additional folders of standard locations still ignored otherwise, exclude folders to customize additional folders for irrelevant data, and ignore list due to not yet proposed update to Windows 10 1903, and to include Microsoft products reporting. Scanning consists of several substeps. Some of these substeps result in changing display while others don't have any effect on the display remaining for longer time unchanged. Why not show some kind of progress bar or progress information?

And shall really all substeps of the scan action be blocking?

Do you consider moving blocking action substeps on a command queue in order to reduce blocking time, including those not requiring blocking but depending on the blocked ones?
Additional InformationThe first steps including the real scan action take about a bit more than 9 out of those 20 minutes with continuous updates of the display showing progress. The remaining almost 11 minutes, a mere "Processing ... Please wait" is displayed the whole time without any change in the display.

Reloading after real scan action substep is considered required and takes about a bit more than 4 out of those 20 minutes without any progress information.

Filtering elements considered unnecessary takes about a bit more than 4 out of those 20 minutes without any progress information.

Filtering elements considered duplicates takes almost 3 out of those 20 minutes without any progress information.

Filtering elements considered useless takes less than a second. What's the difference between unnecessary and useless elements?

Not to forget to report further observed dimensions. During scanning, the counter of detected objects increases unto almost 8500 objects as I don't have installed any development environment and not yet included some system administration tools already installed. These almost 8500 objects are cleaned up to result in almost 1800 components and products being reported. I didn't count nor export the ignore list. According to SUMo log, it has 60 elements.

BTW, why does SUMo log report a cache size of 1423 elements for 1784 elements of its load list and according to which criteria elements are skipped of its cache before scanning?
And why does SUMo log report a cache size of 1434 elements for 1799 elements of its load list and according to which criteria elements are skipped of its cache after scanning and checking?
TagsNo tags attached.

Relationships

related to 0005485 acknowledgedKyle_Katarn "Blocking action" feature still not documented after more than 11 years now 
related to 0005542 acknowledgedKyle_Katarn provide other logging options for verbose mode storage location 

Activities

Kyle_Katarn

2019-06-10 15:07

administrator   ~0003462

Checking operation : Nothing to report, as per design :
Info [19-06-10 12:36:50] Operation successful in 429594 ms
Info [19-06-10 12:36:50] Min : 156 ms / Avg : 193 ms / Max : 485 ms

> This is a rather good average performance
> SUMo not designed for a single PC with 1800 software installed
> I'll work on a more efficient "burst request" later.

Regarding Scan : please provide log file too.
One request : if you empty list and do & Normal scan, how long does it take ?

wolf

2019-06-10 15:57

reporter   ~0003463

The originally attached log file is already for scanning. Nothing is removed of the scanning action. It is standard logging, not verbose logging as verbose logging impacts performance and the expected number of records (17000 - 30000) for verbose mode exceeds the limit of available records (10000) in that location.

Only for the checking action, I removed most content of that log file before attaching, keeping start and end inside.

What do you mean by empty list?
Do you mean to empty the additional folder list temporarily, keeping excluded folder list and ignore list unchanged?
If yes, you mean to repeat the same action with that changed additional folder list, keeping Microsoft product reporting and replace temporarily deep scan by normal scan?

wolf

2019-06-10 16:16

reporter   ~0003465

I just realized its the wrong version of the log file attached. Please replace with the newly attached one.

SUMo.scan.20190610-2.log (7,923 bytes)

Kyle_Katarn

2019-06-10 16:59

administrator   ~0003466

Any reason why c:\texlive takes so much time ? same for E:\Bitnami ?

wolf

2019-06-10 19:51

reporter   ~0003467

Yes.

C:\texlive is my storage location of the TeX Live distribution. This distribution is used by LyX which is a TeX based document editor using a TeX distribution for document processing. The size of my TeX Live distribution has halved as I removed the 2018 version, keeping the 2019 version. That's about 6 GByte in almost 190000 files and almost 13800 directories.

E:\Bitnami is the container for currently one server stack. Bitnami provides several. The one installed is WAMP. That's about 1.1 GByte in almost 41000 files and almost 7000 directories.

Kyle_Katarn

2019-06-10 20:37

administrator   ~0003469

Good reason for SUMo to take time...

I'd suggest, in your case to :
- clear list and revert to normal mode
- manually add the extra files using additional folder for most cases
- manually add the extra files using specific exe file for "large folders"

wolf

2019-06-10 21:56

reporter   ~0003473

I didn't intend this issue to improve performance as performance is addressed with other issues and ideas. And I've added most entries to the additional folders list due to still many standard installation locations not yet investigated by SUMo scan action by default as a work around and hence expecting performance decrease.

And I asked you to remove my first log file of this issue as I don't want so many details published in Internet. The 2nd log file is ok as I've condensed it to the part relevant for this issue.

This issue is about how SUMo GUI handles long running background jobs. There are no indications if SUMo got hung or is still processing as its GUI doesn't change for 11 out of 20 minutes. I propose to either use a progress bar for these long running jobs and resp. or provide further details like the currently processed substep job. But if you decide for these further details, reporting this substeps in the status line will not be sufficient. It may be used as prefix. Additionally, it may need some high level reporting of the area currently processed. Low level reporting would probably further deteriorate performance and would better be reserved for verbose mode only.

Kyle_Katarn

2019-06-10 23:19

administrator   ~0003475

Log file removed.
Good point for GUI and feedback to user on what's going on. To be improved.

Issue History

Date Modified Username Field Change
2019-06-10 14:36 wolf New Issue
2019-06-10 14:36 wolf File Added: SUMo.scan.20190610.log
2019-06-10 14:36 wolf Issue generated from: 0005485
2019-06-10 14:36 wolf Relationship added related to 0005485
2019-06-10 15:07 Kyle_Katarn Assigned To => Kyle_Katarn
2019-06-10 15:07 Kyle_Katarn Status new => feedback
2019-06-10 15:07 Kyle_Katarn Note Added: 0003462
2019-06-10 15:15 wolf Issue cloned: 0005542
2019-06-10 15:15 wolf Relationship added related to 0005542
2019-06-10 15:57 wolf Note Added: 0003463
2019-06-10 16:16 wolf File Added: SUMo.scan.20190610-2.log
2019-06-10 16:16 wolf Note Added: 0003465
2019-06-10 16:59 Kyle_Katarn Note Added: 0003466
2019-06-10 19:51 wolf Note Added: 0003467
2019-06-10 20:37 Kyle_Katarn Note Added: 0003469
2019-06-10 21:56 wolf Note Added: 0003473
2019-06-10 23:19 Kyle_Katarn File Deleted: SUMo.scan.20190610.log
2019-06-10 23:19 Kyle_Katarn Note Added: 0003475
2019-06-10 23:20 Kyle_Katarn Status feedback => acknowledged
2019-06-10 23:20 Kyle_Katarn Target Version => Short term