PDF Export problems caused by upgrading mydbr from 6.6.1 to 6.6.2

(13 posts) (2 voices)

Tags:

No tags yet.

  1. shem, Member

    Hi,
        Today we purchased a full year's OEM license, and to get it to be applied I had to upgrade myDBR from 6.6.1 to 6.6.2
    This has caused problems.

    Please see the email I have sent to myDBR support in which I attached two PDF documents that were downloaded for the same report.
    One from yesterday, July 18th, when we were running myDBR version 6.6.1
    And one from today, July 19th, when we ran myDBR version 6.6.2

    Here are the problems.
    1. The PDF download itself takes much longer.  With 6.6.1 it only took a few seconds.  With 6.6.2 it took over a minute.
    2. As you can see from the two attachments, all my formatting (from  webroot/mydbr/user/userstyle.css) is not being applied in the 6.6.2 version.  Note:  The formatting was being applied on the report itself, before pressing the PDF Export button.
    3. This includes the logo no longer being displayed in the PDF header (which we display using dbr.html - not export_header_pdf.php).  It should be displayed to the left of "Arrow Acceptance, LLC"
    4. Why are the "Parameter" user input text boxes being displayed in the PDF export.  They were not displayed on the report itself before the export, and they should certainly be not be showing when all the input parameters have been filled with input.

    Does version 6.6.2 have new Environment Settings, that need to be populated, in order for things to work the way they did in 6.6.1?
    If there are other changes needed to make the PDF export work exactly the same way it did in 6.6.1 then please tell me very explicitly what to do.

    If there is indeed a problem with version 6.6.2, then is there a way to revert to 6.6.1 even with the new license that we purchased?
    Thanks in advance

  2. shem, Member

    Further to my above question:
    When we purchased the OEM license for our test machine, whose hostname is testb.verifacto.com, what was I suppose to put as the hostname?
    testb.verifacto.com
    or localhost
    ?
    I assume the mydbr database will always be on the same machine as our web server.
    In the file mydbr_config.php
    we have
    define( "DB_HOST", 'localhost' );
    So should I have written localhost as the hostname for the license?
    Could this be the reason why the report looks fine, but PDF export has a problem and takes a long time?
    Or does it not matter what the hostname for the license says.
    The

    If I was suppose to put localhost instead of testb.verifacto.com, how do I change it?

    p.s. In addition to the license for our test machine, we also purchased an OEM license for my local developers virtual machine.
    I also upgraded that to mydbr version 6.6.2
    On my local virtual machine, the PDF Export looks fine and it downloads within seconds.
    The only difference I noticed on my local VM is that when I click on the PDF export button, then even though in my report I have
    IF (inExportFormat = 'pdf') THEN
    select 'dbr.export.options', 'content-disposition.pdf', 'attachment';

    it is BOTH doing an automatic download, AND displaying the PDF export in a new tab.

  3. myDBR Team, Key Master

    1. The PDF download itself takes much longer. With 6.6.1 it only took a few seconds. With 6.6.2 it took over a minute.

    There is no change in exports bwetween 6.6.1 and 6.6.2. They use exactly the same code. Can you check your report/server setup.

    2. As you can see from the two attachments, all my formatting (from webroot/mydbr/user/userstyle.css) is not being applied in the 6.6.2 version. Note: The formatting was being applied on the report itself, before pressing the PDF Export button.
    3. This includes the logo no longer being displayed in the PDF header (which we display using dbr.html - not export_header_pdf.php). It should be displayed to the left of "Arrow Acceptance, LLC"

    There are no attachments in forum. Have you changed your myDBR setup/config files somehow?

    4. Why are the "Parameter" user input text boxes being displayed in the PDF export. They were not displayed on the report itself before the export, and they should certainly be not be showing when all the input parameters have been filled with input.

    Sounds like you have somehow changed the myDBR config files somehow. "Parameter" user input fields are not shown in PDF exports (unless wanted). See a demo report.

    --
    myDBR Team

  4. myDBR Team, Key Master

    When we purchased the OEM license for our test machine, whose hostname is testb.verifacto.com, what was I suppose to put as the hostname?
    testb.verifacto.com or localhost?

    The license hostname should match the hostname in your report URL, i.e. the server where myDBR application is running. The hostname in mydbr_config.php points to your database server which can be on the same server as your PHP installation or on different one. That hostname has nothing to do with the license hostname.

    Could this be the reason why the report looks fine, but PDF export has a problem and takes a long time?

    No, it is not related to license hostname.

    it is BOTH doing an automatic download, AND displaying the PDF export in a new tab.

    Please check the code behind your PDF export button. Sounds like it is doing the export twice.

    --
    myDBR Team

  5. shem, Member

    Ok, so thank you for the answer that
    The license hostname should match the hostname in your report URL
    That answers one question.

    Yes, I know that the forum does not accept attachments. That is why I wrote
    Please see the email I have sent to myDBR support in which I attached two PDF documents that were downloaded for the same report.
    One successful from yesterday, July 18th, when we were running myDBR version 6.6.1
    And one from today, July 19th, when we ran myDBR version 6.6.2 which is messed up.

    But I have not touched the config files
    Regarding mydbr_config.php
    ubuntu@ip-172-16-0-223:~/webroot/mydbr$ ls -l mydbr_config.php
    -rwxr-xr-x 1 www-data ubuntu 333 Jul 16 13:11 mydbr_config.php
    You can see the date is July 16th
    Yesterday July 18th, the PDF Export worked.

    Regarding defaults.php in the mydbr directory
    ubuntu@ip-172-16-0-223:~/webroot/mydbr$ ls -lrt defaults.php
    -rwxr-xr-x 1 www-data ubuntu 35331 Jul 18 10:59 defaults.php
    You can see the date and time is ubuntu@ip-172-16-0-223:~/webroot/mydbr$ ls -lrt defaults.php
    -rwxr-xr-x 1 www-data ubuntu 35331 Jul 18 10:59 defaults.php - This is Eastern time - New York
    and I don't remember touching it anyway, but in any case the successful PDF file which I generated with version 6.1.1 and attached to the email has a date and time of Jul 18 11:11 Eastern time
    which is later.

    And in the mydbr/user directory, all the files are from July 17th or earlier.

    In Summary, I tested my reports successfully last night and did not change any thing after that.
    This morning we purchased the OEM yearly license for our testb server, and all I did after that was go into the updates page and click on Update myDBR to update from 6.6.1 to 6.6.2.
    Now the PDF download is messed up AND it takes four minutes to download.

    This is not acceptable.
    Please help me.
    On my local Virtual machine, the PDF export works and it takes only a few seconds to download. So, if you tell me which config files to check, I will do a file comparison to see if they are different.
    Tell me what else I can check?
    The only environment setting which is different, is that on the testb server we are using SSO authentication, and on my local VM I am not. But I just disabled SSO and the result was the same.
    Are you sure there are no new environment settings with 6.6.2?

  6. myDBR Team, Key Master

    Please see the email I have sent to myDBR support in which I attached two PDF documents that were downloaded for the same report.
    One successful from yesterday, July 18th, when we were running myDBR version 6.6.1
    And one from today, July 19th, when we ran myDBR version 6.6.2 which is messed up.

    Unfortunately we have not received this one. Make sure you attach the files in PDF format.

    As said, the export functionality has not changed between 6.6.1 and 6.6.2. In addition to that, your local export (with latest version) works fine and we do not see any changes in PDF export performance in out setups (including the demo which you can see yourself).

    So, the slowdown is in your second setup.

    You reported multiple problems:

    Slow PDF generation (over a minute/four minutes)

    How long does it take to produce the HTML report and how long does it take the server to convert it to PDF. How big is the file? You can use the export debug option to separate the two of these and see where the slowdown comes from.

    Check first that the server recognizes it's own hostname and is able to access files in it via the URL. This could cause the slowdown and the parameters showing up.

    "Parameter" user input text boxes being displayed in the PDF export

    The parameter input is hidden by default with 'invisible' CSS class. Make sure the PDF file can access the CSS definitions (see the export debug).

    when I click on the PDF export button ... it is BOTH doing an automatic download, AND displaying the PDF export in a new tab.

    What is the "export button" you are referring to? This is probably a separate problem.

    This is not acceptable. Please help me.

    We do understand that you have a problem. The problem is however in your server setup and without access to it, we can only quess what might cause the problems.

    The best quess for now is the server not knowing it's own name / does not recognize the certificate if the server has one.

    --
    myDBR Team

  7. shem, Member

    I just sent the email again to support@mydbr.com with two PDF attachments. They are the exact PDF files that were downloaded by myDBR.
    So please tell me if you got the email. For extra measure I forwarded it to my private Yahoo email, and from there forwarded it back to support@mydbr.com

    Please note this problem happens simply from the myDBR portal itself. I am not introducing a separate complexity of an embedded report.
    The Export button I am talking about is simply the myDBR Portal download arrow which when pressed shows a drop down menu between Excel, PDF, CSV, SQL

    The PDF Export (with 6.6.2) itself takes Four minutes. With 6.6.1 it only took a few seconds. (and the html output takes less than one second).
    So the slowdown, is only as a result of pressing PDF on the myDBR portal's drop down menu.

    You write:
    The parameter input is hidden by default with 'invisible' CSS class. Make sure the PDF file can access the CSS definitions
    How do I "Make sure the PDF file can access the CSS definitions"?
    You then wrote:
    (see the export debug).
    So I followed the instructions, and the end result was an error
    Failed to load http://tmp/mydbr_export_810202457.html, with network status code 3 and http status code 0 - Host tmp not found
    But this seems to be irrelevant because I get a similar error on my local VM where the PDF export works fine.

    Finally, what do you mean by
    the server not knowing it's own name / does not recognize the certificate if the server has one.
    Everything works, except for the format (and slowness) of the PDF export, and even that worked before I pressed the button to update myDBR to 6.6.2

    P.S. The one thing we didn't discuss and might hint at a solution:
    I now see that on the Environment Settings there is a section called OEM.
    Most of the input boxes in that section are blank.
    I never noticed that section before.
    Was I supposed to fill those boxes out, and if so with what values?

    p.s. If all else fails, is there a way to revert to myDBR 6.6.1?

  8. myDBR Team, Key Master

    So I followed the instructions, and the end result was an error
    Failed to load http://tmp/mydbr_export_810202457.html, with network status code 3 and http status code 0 - Host tmp not found
    But this seems to be irrelevant because I get a similar error on my local VM where the PDF export works fine.

    This is highly relevant to your problem and a first conrete step for you to solve the issue. What exact steps did you follow to get this error and where does this error come from?

    We did now receive the PDF exports. Looks like the error above is causing the problem. The server is trying to access the CSS files but fails. Can you send the generated html file (like mydbr_export_810202457.html) that you get when the export debug flag is on.

    I now see that on the Environment Settings there is a section called OEM.
    Most of the input boxes in that section are blank.
    I never noticed that section before.
    Was I supposed to fill those boxes out, and if so with what values?

    You already opened another topic with this question. Please try to keep it in one topic, otherwise the same discussion is on multiple topics. Also, please use either the support email or the forum, not both for the same questions.

  9. shem, Member

    Ok, here is how I generated the above error on our testb server:

    Here is the first portion of my stored procedure

    CREATE PROCEDURE `sp_DBR_assets`(IN clientdb VARCHAR(100),
    IN _location_name VARCHAR(100), IN _portfolio_name VARCHAR(100), IN _status VARCHAR(50), IN _begin_date DATE, IN _end_date DATE,
    inExportFormat VARCHAR(50), inAutoReportProcedure VARCHAR(100))
    BEGIN
    IF (inExportFormat = 'pdf') THEN
    select 'dbr.export.options', 'debug', 1;
    #select 'dbr.export.options', 'content-disposition.pdf', 'attachment';
    select 'dbr.title', '';
    SELECT `name` into @report_name FROM mydbr_reports WHERE proc_name = inAutoReportProcedure;
    select 'dbr.wkhtmltopdf', "--header-html ''"; # Disable default PDF header.
    CALL sp_DBR_pdf_header(clientdb, @report_name);
    END IF;

    As you see I placed the debug statement in the beginning of the IF that handles a PDF export request.
    p.s. I commented out for now the content-disposition because the same problem occurs whether I download it or just export it.

    When I run the report from the myDBR portal the browser displays the following:
    To test wkhtmltopdf pdf generation, copy the following command into the server's command line.
    myDBR saved the generated HTML file into /tmp/mydbr_export_810202457.html. File size is: 97227 bytes.
    $ '/usr/bin/wkhtmltopdf' --margin-top "20" --header-spacing "8" --enable-local-file-access --replace rdate "2023-07-19" --replace rtime "01:57:35 pm" --header-html '' /tmp/mydbr_export_810202457.html debug_output.pdf

    So I opened up a terminal window and command line for the server, went to the webroot directory and entered
    '/usr/bin/wkhtmltopdf' --margin-top "20" --header-spacing "8" --enable-local-file-access --replace rdate "2023-07-19" --replace rtime "01:57:35 pm" --header-html '' /tmp/mydbr_export_810202457.html debug_output.pdf
    The response was
    Loading pages (1/6)
    Error: Failed to load http://tmp/mydbr_export_810202457.html, with network status code 3 and http status code 0 - Host tmp not found
    Error: Failed loading page http:///tmp/mydbr_export_810202457.html (sometimes it will work just to ignore this error with --load-error-handling ignore)
    Exit with code 1 due to network error: HostNotFoundError

    I cannot send you the generated html file mydbr_export_810202457.html
    because it does not exist anywhere on the machine.
    And why is it trying to load http://tmp/mydbr_export_810202457.html
    Does it think that the name of the server is "tmp"?

    Anyway, you maybe correct that this is a lead to solving the problem because while on my local VM, when I ran the same debug test and I got the message Failed to load http://tmp/mydbr_export_464138975.html
    I searched and found a file called
    /tmp/systemd-private-120fd024bf864cf7a44c018a09f3bf4d-apache2.service-bX7uth/tmp/mydbr_export_464138975.html

    However, on the testb server, when I ran the debug, a directory was generated named
    /tmp/systemd-private-9bf2692c915b4543be3195bcb713710d-apache2.service-zWBCvi
    and it generated a subdirectory called "tmp", but that tmp subdirectory was empty.

    So this leaves us with questions:
    1) Why on testb did it fail to generate mydbr_export_810202457.html under the above tmp subdirectory?
    2) What is the connection between this failure, and the problem you state of "the server is trying to access the CSS files but fails." By the way are you referring to my userstyle.css, or to some myDBR internal css files?
    3) Why when I ran the same exact report yesterday (on testb), with myDBR version 6.6.1, did the PDF export work?

    Thank you for your patience, and I apologize for duplicate questions.

  10. shem, Member

    Good morning.
    Looking forward to your response to my last set of questions.

    In the meanwhile I once again ran the debug test that you asked me to run yesterday.
    But before I did that I changed access to the directory to full access
    It had been
    drwx------ 3 root root 4096 Jul 19 14:01 systemd-private-9bf2692c915b4543be3195bcb713710d-apache2.service-zWBCvi
    I changed it to
    drwxrwxrwx 3 root root 4096 Jul 19 14:01 systemd-private-9bf2692c915b4543be3195bcb713710d-apache2.service-zWBCvi
    So now when I run the debug test, I still get the same message
    To test wkhtmltopdf pdf generation, copy the following command into the server's command line.
    myDBR saved the generated HTML file into /tmp/mydbr_export_486385262.html. File size is: 97227 bytes.
    $ '/usr/bin/wkhtmltopdf' --margin-top "20" --header-spacing "8" --enable-local-file-access --replace rdate "2023-07-20" --replace rtime "03:31:08 am" --header-html '' /tmp/mydbr_export_486385262.html debug_output.pdf

    But this time the file mydbr_export_486385262.html was indeed generated.
    ubuntu@ip-172-16-0-223:/tmp$ sudo ls -l systemd-private-9bf2692c915b4543be3195bcb713710d-apache2.service-zWBCvi/tmp
    total 96
    -rw-r--r-- 1 www-data www-data 97227 Jul 20 03:31 mydbr_export_486385262.html

    I opened the file in my chrome browser and it looks as follows:
    The top part is the PDF header that I expect, including the images
    The bottom, i.e. the table grid looks exactly like the HTML generated grid that shows up when I generate the report on the browser.
    And all this happened within 1 second.
    But of course, this doesn't help me with PDF export which still takes four minutes and still had bad formatting.
    In any case since you requested that I send you the generated html, I am sending it to you via email support.

    One more thing:
    Since the above message from wkhtmltopdf suggested to run wkhtmltopdf again with the option with --load-error-handling ignore
    I did so. This time the output was as follows:
    Loading pages (1/6)
    Error: Failed to load http://tmp/mydbr_export_486385262.html, with network status code 3 and http status code 0 - Host tmp not found
    Warning: Failed loading page http:///tmp/mydbr_export_486385262.html (ignored)
    Counting pages (2/6)
    Resolving links (4/6)
    Loading headers and footers (5/6)
    Printing pages (6/6)
    Done
    Exit with code 1 due to network error: HostNotFoundError

  11. myDBR Team, Key Master

    You have two separate issues. One for running the debug command in command line and the other one causing your problem with PDF creation.

    1) When myDBR creates the PDF file, it first creates the file into PHP's tmp directory if not otherwise specified. It then uses wkhtmltopdf to turn it to PDF. If the PHP's tmp directory is other than the real directory name (systemd-private-9bf2692c915b4543be3195bcb713710d-apache2.service-zWBCvi/tmp
    vs /tmp), the debug command does not know it, hence failing to load the file. wkhtmltopdf puts out an uninformative error message http://tmp/mydbr_export_486385262.html while it is trying to access the /tmp/mydbr_export_486385262.html.

    If you want you can configure the wkhtmltopdf to use another directory so that the debug generated command works. See tmp_directory from the docuemntation.

    2) The second one is the one that is causing your problems. When wkhtmltopdf parses the HTML document,it tries to load the JavaScript and CSS files referenced in the HTML file. If your server does not know it's own hostname or if PHP does not recognize the certificate, the requests take a long time eventually timing out and your report is missing the CSS files. Your one does.

    To check if the server knows its hostname you can ping your server (testb.verifacto.com). if this fails, add the definition into the /etc/hosts.

    Check also that the server knows its' own certificate.

    --
    myDBR Team

  12. shem, Member

    Update:
    You were right about the need for the the server to know its hostname
    It now works.
    Thank you so much for your help.

    I have now added to /etc/hosts
    172.16.0.223 testb.verifacto.com
    and ping testb.verifacto.com works
    where
    172.16.0.223 is the internal IP address.

    The only thing I can't figure out, is how did work two days ago, with version 6.6.1, without the entry in /etc/hosts?

    p.s. Something that would be good for the myDBR support team to know:
    When you use the "export debug" option, and run via PDF and the browser shows you instructions on how to test the PDF creation, the instructions are misleading:
    1) it implies that the html file it says to use as input (to the wkhtmltopdf command) is under /tmp
    In fact its under something like /tmp/systemd-private-9bf2692c915b4543be3195bcb713710d-apache2.service-1yzILh/tmp/
    2) It neglects to tell you to run wkhtmltopdf (as a file and not a URL) with a prefix of file://
    3) It neglects to tell you that both
    /tmp/systemd-private-9bf2692c915b4543be3195bcb713710d-apache2.service-1yzILh/
    and
    /tmp/systemd-private-9bf2692c915b4543be3195bcb713710d-apache2.service-1yzILh/tmp/
    need to have permissions that wkhtmltopdf can access namely eXecute permission.

  13. myDBR Team, Key Master

    The only thing I can't figure out, is how did work two days ago, with version 6.6.1, without the entry in /etc/hosts?

    As previously stated, there was no change in exports between those days or versions. It is possible that your server configuration may have been altered or something else at your end was changed.

    p.s. Something that would be good for the myDBR support team to know:
    When you use the "export debug" option, and run via PDF and the browser shows you instructions on how to test the PDF creation, the instructions are misleading:
    1) it implies that the html file it says to use as input (to the wkhtmltopdf command) is under /tmp
    In fact its under something like /tmp/systemd-private-9bf2692c915b4543be3195bcb713710d-apache2.service-1yzILh/tmp/

    This is a server configuration issue. myDBR uses the 'tmp' directory by default, but your server uses a private temp directory that is invisible to PHP and, consequently, to myDBR.

    2) It neglects to tell you to run wkhtmltopdf (as a file and not a URL) with a prefix of file://

    There is no need to use the 'file://' prefix. The wkhtmltopdf error message erroneously adds 'http:/' to a file parameter.

    3) It neglects to tell you that both
    /tmp/systemd-private-9bf2692c915b4543be3195bcb713710d-apache2.service-1yzILh/
    and
    /tmp/systemd-private-9bf2692c915b4543be3195bcb713710d-apache2.service-1yzILh/tmp/
    need to have permissions that wkhtmltopdf can access namely eXecute permission.

    This issue is once again related to your server's private tmp configuration, which remains invisible to PHP. When debugging wkhtmltopdf, instead of modifying permissions for the private tmp, consider moving the HTML file from the private tmp directory to /tmp.

    Please note that the use of private tmp is beneficial and will work perfectly fine with PDF exports. The only problem arises when PHP cannot detect the utilization of private tmp while creating the wkhtmltopdf command.

    --
    myDBR Team


Reply

You must log in to post.