Discussion:
QxCompiler - add ES6, faster compilation, and 100% Javascript API to building applications
(too old to reply)
John Spackman
2016-02-13 21:02:18 UTC
Permalink
Hi all

There is a first release of my QxCompiler that adds ES6 to Qooxdoo applications and replaces the generate.py toolchain with a faster, 100% Javascript tool that is easily extensible.

You can find the first release at GitHub here: https://github.com/johnspackman/qxcompiler

It’s an alpha release, but something that’s been in development for a while now and which I’m starting to build into my production servers.

I’m very open to pull requests or collaboration, and keen to see this become a useful tool for myself as well as others so any questions etc please ask.

Regards
John
Dimitri
2016-02-14 02:14:59 UTC
Permalink
John, congratulations with the long awaited release! :)

qooxdoo guys, do you think that project like this could at some moment
land in qooxdoo and get official support? What about qooxdoo patches
(mostly strict mode compatibility related, AFAIK) - could they be
merged upstream? This would reduce maintainship costs for those who
want to experiment with both QxCompiler and official toolchain.

I didn't yet try QxCompiler with my project - I feel I yet lack some
understanding of how it works. However, I tried to test it with the
skeleton app. Off the top of my head:

- one needs to do "npm install" first and to run test scripts with
"node <script>.js". This might be quite obvious for those experienced
with Node, but I guess the docs will nevertheless benefit from
mentioning this;
- it's not easy to clone a repo unless you've set up Github SSH access.
This is because of "qooxdoo" submodule pointing to "***@github.com:john
spackman/qooxdoo.git". Could it be a HTTPS URL instead?
- does QxCompiler honour QOOXDOO_PATH setting? How do I define qooxdoo
location if it is different from the bundled one?
- testdata/qxt directory doesn't contain skeleton app. In order to play
with test scripts, one needs to create the app manually (as
"skeleton.Application"?) and copy it to the said location;
- does lib/compiler.jar really belong there?

Most importantly, I was unable to find clear instructions how to
produce a single-script minified build (a-la "generate.py build"). By
deafult, QxCompiler produces a bunch of JS files, which significantly
slows down loading and, obviously, is not acceptable for production. Is
it possible at all with the current version (maybe in combination with
classic generator)?

I hope these are just minor issues. After all, you've done a great job
:) I wish you good luck and further progress wih QxCompiler.

Dimitri
Post by John Spackman
Hi all
There is a first release of my QxCompiler that adds ES6 to Qooxdoo
applications and replaces the generate.py toolchain with a faster,
100% Javascript tool that is easily extensible.
You can find the first release at GitHub here: https://github.com/joh
nspackman/qxcompiler
It’s an alpha release, but something that’s been in development for a
while now and which I’m starting to build into my production
servers.  
I’m very open to pull requests or collaboration, and keen to see this
become a useful tool for myself as well as others so any questions
etc please ask.
Regards
John
-------------------------------------------------------------------
-----------
Site24x7 APM Insight: Get Deep Visibility into Application
Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
John Spackman
2016-02-14 09:49:03 UTC
Permalink
Hi Dimitri

Thanks for checking out so quickly :)

Re: npm and docs: done

Re: qooxdoo submodule and https: fixed

Re: QOOXDOO_PATH: no - that's a generator.py/config.json setting and config.json files are completely ignored; when using the QxCompiler API, once you have told it where the Qooxdoo library is you have done the equivalent of setting QOOXDOO_PATH. Of course, as it's API based you can define variables etc as suits you best :)

In test/compile-app-demo.js it adds the Qooxdoo library like this:
maker.addLibrary("../qooxdoo/framework", cb);

Just change the path to point to your preferred location for qooxdoo. To make this a bit clearer, I've just modified the test/compile-app-demo.js to declare a variable QOOXDOO_PATH

I expect that the command line version will imply have a search path for libraries and just auto-discover by searching for Manifest.json files.

Re: missing testdata/qxt: something went wrong with your checkout, that directory is definitely there.

Re: build (and source-hybrid) targets: to control output for Source vs Build vs Source-Hybrid, the API classes qxcompiler.targets.SourceTarget, qxcompiler.targets.BuildTarget, and qxcompiler.targets.SourceHybridTarget are used; test/compile-app-demo.js uses the SourceTarget but you should be able to switch it easily enough. I say "should" because I realise now that I havn't tested them for a few weeks now and I should give them the once over ASAP :) I've got to go out shortly so I'll take a look this evening.

John



----------------------------------------
From: "Dimitri" <***@cargosoft.ru>
Sent: Sunday, February 14, 2016 2:27 AM
To: "qooxdoo Development" <qooxdoo-***@lists.sourceforge.net>
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster compilation, and 100% Javascript API to building applications
John, congratulations with the long awaited release! :) qooxdoo guys, do you think that project like this could at some moment land in qooxdoo and get official support? What about qooxdoo patches (mostly strict mode compatibility related, AFAIK) - could they be merged upstream? This would reduce maintainship costs for those who want to experiment with both QxCompiler and official toolchain. I didn't yet try QxCompiler with my project - I feel I yet lack some understanding of how it works. However, I tried to test it with the skeleton app. Off the top of my head: - one needs to do "npm install" first and to run test scripts with "node <script>.js". This might be quite obvious for those experienced with Node, but I guess the docs will nevertheless benefit from mentioning this; - it's not easy to clone a repo unless you've set up Github SSH access. This is because of "qooxdoo" submodule pointing to "***@github.com:john spackman/qooxdoo.git". Could it be a HTTPS URL instead? - does QxCompiler honour QOOXDOO_PATH setting? How do I define qooxdoo location if it is different from the bundled one? - testdata/qxt directory doesn't contain skeleton app. In order to play with test scripts, one needs to create the app manually (as "skeleton.Application"?) and copy it to the said location; - does lib/compiler.jar really belong there? Most importantly, I was unable to find clear instructions how to produce a single-script minified build (a-la "generate.py build"). By deafult, QxCompiler produces a bunch of JS files, which significantly slows down loading and, obviously, is not acceptable for production. Is it possible at all with the current version (maybe in combination with classic generator)? I hope these are just minor issues. After all, you've done a great job :) I wish you good luck and further progress wih QxCompiler. Dimitri > Hi all > > There is a first release of my QxCompiler that adds ES6 to Qooxdoo > applications and replaces the generate.py toolchain with a faster, > 100% Javascript tool that is easily extensible. > > You can find the first release at GitHub here: https://github.com/joh > nspackman/qxcompiler > > It's an alpha release, but something that's been in development for a > while now and which I'm starting to build into my production > servers. > > I'm very open to pull requests or collaboration, and keen to see this > become a useful tool for myself as well as others so any questions > etc please ask. > > Regards > John > > > > > > ------------------------------------------------------------------- > ----------- > Site24x7 APM Insight: Get Deep Visibility into Application > Performance > APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month > Monitor end-to-end web transactions and take corrective actions now > Troubleshoot faster and improve end-user experience. Signup Now! > http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140 > _______________________________________________ > qooxdoo-devel mailing list > qooxdoo-***@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel ------------------------------------------------------------------------------ Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140 _______________________________________________ qooxdoo-devel mailing list qooxdoo-***@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
John Spackman
2016-02-14 10:37:47 UTC
Permalink
Further to this - I've added some docs about the API, but also seen that the BuildTarget has a bug and doesn't work - it's only a minor issue but I probably wont have time to fix it until tonight

John




----------------------------------------
From: "John Spackman" <***@zenesis.com>
Sent: Sunday, February 14, 2016 9:50 AM
To: "qooxdoo Development" <qooxdoo-***@lists.sourceforge.net>
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster compilation, and 100% Javascript API to building applications
Hi Dimitri

Thanks for checking out so quickly :)

Re: npm and docs: done

Re: qooxdoo submodule and https: fixed

Re: QOOXDOO_PATH: no - that's a generator.py/config.json setting and config.json files are completely ignored; when using the QxCompiler API, once you have told it where the Qooxdoo library is you have done the equivalent of setting QOOXDOO_PATH. Of course, as it's API based you can define variables etc as suits you best :)

In test/compile-app-demo.js it adds the Qooxdoo library like this:
maker.addLibrary("../qooxdoo/framework", cb);

Just change the path to point to your preferred location for qooxdoo. To make this a bit clearer, I've just modified the test/compile-app-demo.js to declare a variable QOOXDOO_PATH

I expect that the command line version will imply have a search path for libraries and just auto-discover by searching for Manifest.json files.

Re: missing testdata/qxt: something went wrong with your checkout, that directory is definitely there.

Re: build (and source-hybrid) targets: to control output for Source vs Build vs Source-Hybrid, the API classes qxcompiler.targets.SourceTarget, qxcompiler.targets.BuildTarget, and qxcompiler.targets.SourceHybridTarget are used; test/compile-app-demo.js uses the SourceTarget but you should be able to switch it easily enough. I say "should" because I realise now that I havn't tested them for a few weeks now and I should give them the once over ASAP :) I've got to go out shortly so I'll take a look this evening.

John



----------------------------------------
From: "Dimitri" <***@cargosoft.ru>
Sent: Sunday, February 14, 2016 2:27 AM
To: "qooxdoo Development" <qooxdoo-***@lists.sourceforge.net>
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster compilation, and 100% Javascript API to building applications
John, congratulations with the long awaited release! :) qooxdoo guys, do you think that project like this could at some moment land in qooxdoo and get official support? What about qooxdoo patches (mostly strict mode compatibility related, AFAIK) - could they be merged upstream? This would reduce maintainship costs for those who want to experiment with both QxCompiler and official toolchain. I didn't yet try QxCompiler with my project - I feel I yet lack some understanding of how it works. However, I tried to test it with the skeleton app. Off the top of my head: - one needs to do "npm install" first and to run test scripts with "node <script>.js". This might be quite obvious for those experienced with Node, but I guess the docs will nevertheless benefit from mentioning this; - it's not easy to clone a repo unless you've set up Github SSH access. This is because of "qooxdoo" submodule pointing to "***@github.com:john spackman/qooxdoo.git". Could it be a HTTPS URL instead? - does QxCompiler honour QOOXDOO_PATH setting? How do I define qooxdoo location if it is different from the bundled one? - testdata/qxt directory doesn't contain skeleton app. In order to play with test scripts, one needs to create the app manually (as "skeleton.Application"?) and copy it to the said location; - does lib/compiler.jar really belong there? Most importantly, I was unable to find clear instructions how to produce a single-script minified build (a-la "generate.py build"). By deafult, QxCompiler produces a bunch of JS files, which significantly slows down loading and, obviously, is not acceptable for production. Is it possible at all with the current version (maybe in combination with classic generator)? I hope these are just minor issues. After all, you've done a great job :) I wish you good luck and further progress wih QxCompiler. Dimitri > Hi all > > There is a first release of my QxCompiler that adds ES6 to Qooxdoo > applications and replaces the generate.py toolchain with a faster, > 100% Javascript tool that is easily extensible. > > You can find the first release at GitHub here: https://github.com/joh > nspackman/qxcompiler > > It's an alpha release, but something that's been in development for a > while now and which I'm starting to build into my production > servers. > > I'm very open to pull requests or collaboration, and keen to see this > become a useful tool for myself as well as others so any questions > etc please ask. > > Regards > John > > > > > > ------------------------------------------------------------------- > ----------- > Site24x7 APM Insight: Get Deep Visibility into Application > Performance > APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month > Monitor end-to-end web transactions and take corrective actions now > Troubleshoot faster and improve end-user experience. Signup Now! > http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140 > _______________________________________________ > qooxdoo-devel mailing list > qooxdoo-***@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel ------------------------------------------------------------------------------ Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140 _______________________________________________ qooxdoo-devel mailing list qooxdoo-***@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
Dimitri
2016-02-15 03:28:22 UTC
Permalink
Hi John! Ready for another dozen of questions and bug reports? ;)
1. As for testdata/qxt directory - yup, it is indeed present, but I don't see "source" subdirectory, where I would expect the actual application code to reside. So I just ran "create-application.py -n qxt" in another location and copied the generated "source" subdirectory to testdata/qxt. Is it how it is supposed to work?
2. The build process spits a lot of trace messages like this:
Function enter: anonymous
Function exit: anonymous

SourceTarget completes successfully, but BuildTarget fails with the following:

2016-02-15 04:10:22.045 [info ] makers Writing target Build Target: ../testdata/qxt/build-output/
events.js:141
throw er; // Unhandled 'error' event
^

Error: EMFILE: too many open files, open '../testdata/qxt/build-output//resource/qx/icon/Tango/48/actions/go-home.png'
at Error (native)

I had to increase open file descriptors limit (ulimit -n 8192 worked
for me). If this cannot be fixed/optimized, probably it should be
reflected in the docs.
3. There seems to be a big mess with resources. Test app works well in
its source variant, but the single-script build fails to load an icon:
GET http://localhost:8282/qxcompiler/testdata/qxt/build-output/qxt/test
.png [HTTP/1.0 404 File not found 2ms]
000472 qx.ui.basic.Image[45-0]: Image could not be loaded: qxt/test.png
boot.js:54698:13
000474 qx.ui.basic.Image[45-0]: Image could not be loaded: qxt/test.png
boot.js:54698:13
000569 ImageLoader: Not recognized format of external image
'qxt/test.png'! boot.js:54698:13
Shouldn't it look in "resources" subdirectory instead? The subdir is
present and does contain resources; however, it seems to contain *all*
the available qooxdoo+app resources (~3600 files, ~25M). generate.py
usually produces a ~150K subdir containing resources that are actually
required.
At the same time, my application was unable to load resources neither
in source nor in build variant. In both cases it tried to
(unsuccessfully) look up resources in that very same "source-output" or
"build-output" directory.
4. Our project uses your (excellent :) UploadMgr contrib. I've added
the following to the async.series block:
function(cb) {
maker.addLibrary("/path/to/qooxdoo-contrib/UploadMgr", cb);
},
Is this the right way to integrate a contrib into the project?
Can't yet say whether it worked or not, as I didn't manage to make
application work (see below). I've noticed the following in console:
SyntaxError: in strict mode code, functions may be declared only at top
level or immediately within another function (source-
output/transpiled//com/zenesis/qx/upload/XhrHandler.js:212:19)
(original file line 212, function sendAsMime definition)
5. One of my classes invokes its static method from a constructor.
According to qooxdoo docs, this should be done this way:
this.self(arguments).foo();

The application failed to load because of "this.constructor.self" being
undefined. The following however worked fine:
this.constructor.foo();

This could potentially break applications that access static members
from constructors and/or instance methods. (I'm not sure if constructor
is some special case.)
6. Is there a way to pass qooxdoo environment keys to the app a la
config.js? Much better if the values could be passed via command line.
Our build system produces metadata (version, build number, revision,
timestamp) which we are planning to pass to the app in the form of
qx.core.Environment keys.
7. What about locales & translation? For us, it is essential because
our app is already bilingual, and more languages are to come.
8. For our app, "generate.py build" produces a ~900K script, and it
takes less than a second for the app to be up and running.
QxCompiler/BuildTarget produces a huge ~5M single script, and
application startup takes up to 10 seconds (!)
What is the recommended way to produce optimized/minified build that
would be comparable to the original one?
Okay, that's enough for today :) Tomorrow I'll try to figure out the
cause of another failure that prevents our app from starting up. It's
something related to JSON unmarshalling; the data is unmarshalled as
auto-generated class (qx.data.model.prop1"prop2"prop3[123-0]), while it
should be an application class resolved by a marshal delegate. This
works fine with generate.py.
Good luck!
Dimitri
Post by John Spackman
Further to this - I've added some docs about the API, but also seen
that the BuildTarget has a bug and doesn't work - it's only a minor
issue but I probably wont have time to fix it until tonight
 
John
 
 
 
Sent: Sunday, February 14, 2016 9:50 AM
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster
compilation, and 100% Javascript API to building applications
 
Hi Dimitri
 
Thanks for checking out so quickly :)
 
Re: npm and docs: done
 
Re: qooxdoo submodule and https: fixed
 
Re: QOOXDOO_PATH: no - that's a generator.py/config.json setting and
config.json files are completely ignored; when using the QxCompiler
API, once you have told it where the Qooxdoo library is you have done
the equivalent of setting QOOXDOO_PATH.  Of course, as it's API based
you can define variables etc as suits you best :)
 
    maker.addLibrary("../qooxdoo/framework", cb);
 
Just change the path to point to your preferred location for qooxdoo.
 To make this a bit clearer, I've just modified the test/compile-app-
demo.js to declare a variable QOOXDOO_PATH
 
I expect that the command line version will imply have a search path
for libraries and just auto-discover by searching for Manifest.json
files.
 
Re: missing testdata/qxt: something went wrong with your checkout,
that directory is definitely there.
 
Re: build (and source-hybrid) targets: to control output for Source
vs Build vs Source-Hybrid, the API classes
qxcompiler.targets.SourceTarget, qxcompiler.targets.BuildTarget, and
qxcompiler.targets.SourceHybridTarget are used; test/compile-app-
demo.js uses the SourceTarget but you should be able to switch it
easily enough.  I say "should" because I realise now that I havn't
tested them for a few weeks now and I should give them the once over
ASAP :)  I've got to go out shortly so I'll take a look this evening.
 
John
 
 
Sent: Sunday, February 14, 2016 2:27 AM
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster
compilation, and 100% Javascript API to building applications
 
John, congratulations with the long awaited release! :) qooxdoo guys,
do you think that project like this could at some moment land in
qooxdoo and get official support? What about qooxdoo patches (mostly
strict mode compatibility related, AFAIK) - could they be merged
upstream? This would reduce maintainship costs for those who want to
experiment with both QxCompiler and official toolchain. I didn't yet
try QxCompiler with my project - I feel I yet lack some understanding
of how it works. However, I tried to test it with the skeleton app.
Off the top of my head: - one needs to do "npm install" first and to
run test scripts with "node .js". This might be quite obvious
for those experienced with Node, but I guess the docs will
nevertheless benefit from mentioning this; - it's not easy to clone a
repo unless you've set up Github SSH access. This is because of
spackman/qooxdoo.git". Could it be a HTTPS URL instead? - does
QxCompiler honour QOOXDOO_PATH setting? How do I define qooxdoo
location if it is different from the bundled one? - testdata/qxt
directory doesn't contain skeleton app. In order to play with test
scripts, one needs to create the app manually (as
"skeleton.Application"?) and copy it to the said location; - does
lib/compiler.jar really belong there? Most importantly, I was unable
to find clear instructions how to produce a single-script minified
build (a-la "generate.py build"). By deafult, QxCompiler produces a
bunch of JS files, which significantly slows down loading and,
obviously, is not acceptable for production. Is it possible at all
with the current version (maybe in combination with classic
generator)? I hope these are just minor issues. After all, you've
done a great job :) I wish you good luck and further progress wih
QxCompiler. Dimitri > Hi all > > There is a first release of my
QxCompiler that adds ES6 to Qooxdoo > applications and replaces the
generate.py toolchain with a faster, > 100% Javascript tool that is
https://github.com/joh > nspackman/qxcompiler > > It’s an alpha
release, but something that’s been in development for a > while now
and which I’m starting to build into my production > servers.   > >
I’m very open to pull requests or collaboration, and keen to see this
Post by John Spackman
become a useful tool for myself as well as others so any questions
etc please ask. > > Regards > John > > > > > > --------------------
----------------------------------------------- > ----------- >
Site24x7 APM Insight: Get Deep Visibility into Application >
Performance > APM + Mobile APM + RUM: Monitor 3 App instances at just
$35/Month > Monitor end-to-end web transactions and take corrective
actions now > Troubleshoot faster and improve end-user experience.
Signup Now! > http://pubads.g.doubleclick.net/gampad/clk?id=272487151
&iu=/4140 > _______________________________________________ >
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel ----------
--------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application
Performance APM + Mobile APM + RUM: Monitor 3 App instances at just
$35/Month Monitor end-to-end web transactions and take corrective
actions now Troubleshoot faster and improve end-user experience.
Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________ qooxdoo-devel mailing
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
 ------------------------------------------------------------------
------------
Site24x7 APM Insight: Get Deep Visibility into Application
Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
John Spackman
2016-02-15 12:26:12 UTC
Permalink
HI Dimitri

Ready and waiting ;)

Re 1 testdata/qxt: fixed now, but what you did is fine (my version just has a couple of irrelevant test/demo tweaks)

Re 2 build process, messages: fixed

Re 2 build process, too many open files: ah – OK I can throttle that, I just saw that on my mac ulimit is unlimited (!)

Re 3 icon resource not found: when you copied the source directory into testdata/qxt did you copy the resource directory also? I don’t understand this because it works for me :( You’re right that it should be looking in the resource/ folder for resources, and that’s what mine’s doing here. I’ve just done a fresh checkout from the repo and tried it OK, so please can you try pulling, deleting the build-output directory and try again?

Re 3 all resources being copied: fixed (this should fix your ulimit issue too)

Re 4 UploadMgr: yes that’s exactly how to add a contrib. I’ve fixed the problem in UploadMgr so that it is not strict mode compatible and added it to the qxt app as a demo.

Re 5 this.self() not working: fixed

Re 6 environment settings: yes, I’ve added a property to qxcompiler.Maker that is a map you can set; test/compile-app-demo.js now looks like this:
// Makers use an Analyser to figure out what the Target should write
var maker = new qxcompiler.makers.SimpleApp("qxt.Application", "qxt.theme.Theme").set({
// Targets know how to output an application
target: new qxcompiler.targets.SourceTarget("../testdata/qxt/source-output")
});
maker.setEnvironment({
"qxt.customEnvironment": "this is custom (source target)"
});
Re 7: Locales and translation: Locales are supported but translations not yet; it should be straightforward though, so I’ll take a look at translations maybe tonight, probably tomorrow

Re 8: Large build-target output: the output isn’t compressed/minified yet, but I’ll be adding that in “real soon now”; I’m planning on using UglifyJS to do this, my quick test so far shows that the size of the qxt app is reduced from 3.9Mb to 1.1Mb, so not as good yet as generate.py but in the general area. I think the remaining reduction in size will come by dead code elimination because the superfluous qx.debug code is not removed from the Build Target yet

Well I’m pleased with that for a morning’s work :) There’s a bit more to do that I’ll get onto tonight/tomorrow but it should be enough to get you going on the next step. Don’t forget to checkout the qooxdoo repo as well as the main QxCompiler one.

Cheers
John


From: Dimitri <***@cargosoft.ru>
Reply-To: qooxdoo Development <qooxdoo-***@lists.sourceforge.net>
Date: Monday, 15 February 2016 at 03:28
To: qooxdoo Development <qooxdoo-***@lists.sourceforge.net>
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster compilation, and 100% Javascript API to building applications

Hi John! Ready for another dozen of questions and bug reports? ;)

1. As for testdata/qxt directory - yup, it is indeed present, but I don't see "source" subdirectory, where I would expect the actual application code to reside. So I just ran "create-application.py -n qxt" in another location and copied the generated "source" subdirectory to testdata/qxt. Is it how it is supposed to work?

2. The build process spits a lot of trace messages like this:
Function enter: anonymous
Function exit: anonymous

SourceTarget completes successfully, but BuildTarget fails with the following:

2016-02-15 04:10:22.045 [info ] makers Writing target Build Target: ../testdata/qxt/build-output/
events.js:141
throw er; // Unhandled 'error' event
^

Error: EMFILE: too many open files, open '../testdata/qxt/build-output//resource/qx/icon/Tango/48/actions/go-home.png'
at Error (native)

I had to increase open file descriptors limit (ulimit -n 8192 worked for me). If this cannot be fixed/optimized, probably it should be reflected in the docs.

3. There seems to be a big mess with resources. Test app works well in its source variant, but the single-script build fails to load an icon:

GET Loading Image... [HTTP/1.0 404 File not found 2ms]
000472 qx.ui.basic.Image[45-0]: Image could not be loaded: qxt/test.png boot.js:54698:13
000474 qx.ui.basic.Image[45-0]: Image could not be loaded: qxt/test.png boot.js:54698:13
000569 ImageLoader: Not recognized format of external image 'qxt/test.png'! boot.js:54698:13

Shouldn't it look in "resources" subdirectory instead? The subdir is present and does contain resources; however, it seems to contain *all* the available qooxdoo+app resources (~3600 files, ~25M). generate.py usually produces a ~150K subdir containing resources that are actually required.

At the same time, my application was unable to load resources neither in source nor in build variant. In both cases it tried to (unsuccessfully) look up resources in that very same "source-output" or "build-output" directory.

4. Our project uses your (excellent :) UploadMgr contrib. I've added the following to the async.series block:

function(cb) {
maker.addLibrary("/path/to/qooxdoo-contrib/UploadMgr", cb);
},

Is this the right way to integrate a contrib into the project?

Can't yet say whether it worked or not, as I didn't manage to make application work (see below). I've noticed the following in console:
SyntaxError: in strict mode code, functions may be declared only at top level or immediately within another function (source-output/transpiled//com/zenesis/qx/upload/XhrHandler.js:212:19)
(original file line 212, function sendAsMime definition)

5. One of my classes invokes its static method from a constructor. According to qooxdoo docs, this should be done this way:
this.self(arguments).foo();

The application failed to load because of "this.constructor.self" being undefined. The following however worked fine:
this.constructor.foo();

This could potentially break applications that access static members from constructors and/or instance methods. (I'm not sure if constructor is some special case.)

6. Is there a way to pass qooxdoo environment keys to the app a la config.js? Much better if the values could be passed via command line. Our build system produces metadata (version, build number, revision, timestamp) which we are planning to pass to the app in the form of qx.core.Environment keys.

7. What about locales & translation? For us, it is essential because our app is already bilingual, and more languages are to come.

8. For our app, "generate.py build" produces a ~900K script, and it takes less than a second for the app to be up and running.
QxCompiler/BuildTarget produces a huge ~5M single script, and application startup takes up to 10 seconds (!)
What is the recommended way to produce optimized/minified build that would be comparable to the original one?

Okay, that's enough for today :) Tomorrow I'll try to figure out the cause of another failure that prevents our app from starting up. It's something related to JSON unmarshalling; the data is unmarshalled as auto-generated class (qx.data.model.prop1"prop2"prop3[123-0]), while it should be an application class resolved by a marshal delegate. This works fine with generate.py.

Good luck!
Dimitri

Further to this - I've added some docs about the API, but also seen that the BuildTarget has a bug and doesn't work - it's only a minor issue but I probably wont have time to fix it until tonight

John



From: "John Spackman" <***@zenesis.com>
Sent: Sunday, February 14, 2016 9:50 AM
To: "qooxdoo Development" <qooxdoo-***@lists.sourceforge.net>
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster compilation, and 100% Javascript API to building applications

Hi Dimitri



Thanks for checking out so quickly :)



Re: npm and docs: done



Re: qooxdoo submodule and https: fixed



Re: QOOXDOO_PATH: no - that's a generator.py/config.json setting and config.json files are completely ignored; when using the QxCompiler API, once you have told it where the Qooxdoo library is you have done the equivalent of setting QOOXDOO_PATH. Of course, as it's API based you can define variables etc as suits you best :)



In test/compile-app-demo.js it adds the Qooxdoo library like this:

maker.addLibrary("../qooxdoo/framework", cb);



Just change the path to point to your preferred location for qooxdoo. To make this a bit clearer, I've just modified the test/compile-app-demo.js to declare a variable QOOXDOO_PATH



I expect that the command line version will imply have a search path for libraries and just auto-discover by searching for Manifest.json files.



Re: missing testdata/qxt: something went wrong with your checkout, that directory is definitely there.



Re: build (and source-hybrid) targets: to control output for Source vs Build vs Source-Hybrid, the API classes qxcompiler.targets.SourceTarget, qxcompiler.targets.BuildTarget, and qxcompiler.targets.SourceHybridTarget are used; test/compile-app-demo.js uses the SourceTarget but you should be able to switch it easily enough. I say "should" because I realise now that I havn't tested them for a few weeks now and I should give them the once over ASAP :) I've got to go out shortly so I'll take a look this evening.



John





From: "Dimitri" <***@cargosoft.ru>
Sent: Sunday, February 14, 2016 2:27 AM
To: "qooxdoo Development" <qooxdoo-***@lists.sourceforge.net>
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster compilation, and 100% Javascript API to building applications

John, congratulations with the long awaited release! :) qooxdoo guys, do you think that project like this could at some moment land in qooxdoo and get official support? What about qooxdoo patches (mostly strict mode compatibility related, AFAIK) - could they be merged upstream? This would reduce maintainship costs for those who want to experiment with both QxCompiler and official toolchain. I didn't yet try QxCompiler with my project - I feel I yet lack some understanding of how it works. However, I tried to test it with the skeleton app. Off the top of my head: - one needs to do "npm install" first and to run test scripts with "node <script>.js". This might be quite obvious for those experienced with Node, but I guess the docs will nevertheless benefit from mentioning this; - it's not easy to clone a repo unless you've set up Github SSH access. This is because of "qooxdoo" submodule pointing to "***@github.com:john spackman/qooxdoo.git". Could it be a HTTPS URL instead? - does QxCompiler honour QOOXDOO_PATH setting? How do I define qooxdoo location if it is different from the bundled one? - testdata/qxt directory doesn't contain skeleton app. In order to play with test scripts, one needs to create the app manually (as "skeleton.Application"?) and copy it to the said location; - does lib/compiler.jar really belong there? Most importantly, I was unable to find clear instructions how to produce a single-script minified build (a-la "generate.py build"). By deafult, QxCompiler produces a bunch of JS files, which significantly slows down loading and, obviously, is not acceptable for production. Is it possible at all with the current version (maybe in combination with classic generator)? I hope these are just minor issues. After all, you've done a great job :) I wish you good luck and further progress wih QxCompiler. Dimitri > Hi all > > There is a first release of my QxCompiler that adds ES6 to Qooxdoo > applications and replaces the generate.py toolchain with a faster, > 100% Javascript tool that is easily extensible. > > You can find the first release at GitHub here: https://github.com/joh > nspackman/qxcompiler > > It’s an alpha release, but something that’s been in development for a > while now and which I’m starting to build into my production > servers. > > I’m very open to pull requests or collaboration, and keen to see this > become a useful tool for myself as well as others so any questions > etc please ask. > > Regards > John > > > > > > ------------------------------------------------------------------- > ----------- > Site24x7 APM Insight: Get Deep Visibility into Application > Performance > APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month > Monitor end-to-end web transactions and take corrective actions now > Troubleshoot faster and improve end-user experience. Signup Now! > http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140 > _______________________________________________ > qooxdoo-devel mailing list > qooxdoo-***@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel ------------------------------------------------------------------------------ Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140 _______________________________________________ qooxdoo-devel mailing list qooxdoo-***@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
Dimitri
2016-02-15 18:16:19 UTC
Permalink
Hi! Tried again - things are much better now :) However, two issues
still remain, one of them being resource management.
Indeed, test app loads its icons correctly in both versions.
Unfortunately, this is not the case for my app. What I've found yet:
- the app loads all the qx.* resources correctly (window
minimize/maximize/close icons etc.);
- none of the application's own resources could be loaded. The request
is made to "build-output/myapp/foo.png", which obviously results in a
404.
- in resource-db.json, all resources that *do* load contain non-empty
metadata block, while those that do *not* have an empty literal {}.
The same could be observed in qxt for com/zenesis/qx/upload/*
resources.
(Please note that our app contains not only images, but audio files in
different formats, mainly mp3/ogg. We put them into "resource/sounds".)
The app still cannot load. That unmarshalling error seems to have
gone, but now I've got a "too much recursion" somewhere around the
same location in code. (stacktrace)
I'll try to produce MWEs for both issues later today. Some other stuff
to consider:
1. NodeJS <= 0.12 is not supported, while 4.x and 5.x are fine (due to
object literal extension used at lib/qxcompiler/ClassFile.js:329).
Some distros still ship NodeJS 0.x with their stable versions. I think
this should be mentioned in the docs. OTOH, why not babelize
QxCompiler itself? :)
2. What practice would you recommend for using QxCompiler on a build
server? For each build, a clean checkout is made. Doing npm install in
the project directory is very expensive (~15MB deps). Assuming
QxCompiler resides in its own directory, what would you recommend?
symlink <project>/node_modules -> <QxCompiler>/node_modules? install
deps with npm -g and use linklocal ? Ideally, I'd like some practice
that wouldn't change when we finally have a NPM distro of QxCompiler.
3. It's worth mentioning in the docs that ImageMagick is required (due
to "identify" binary being used in qxcompiler/ResourceManager.js).
Cheers,
Dimitri
Post by John Spackman
HI Dimitri
Ready and waiting ;)
Re 1 testdata/qxt: fixed now, but what you did is fine (my version
just has a couple of irrelevant test/demo tweaks)
Re 2 build process, messages: fixed
Re 2 build process, too many open files: ah – OK I can throttle
that, I just saw that on my mac ulimit is unlimited (!)
Re 3 icon resource not found: when you copied the source directory
into testdata/qxt did you copy the resource directory also? I don’t
understand this because it works for me :( You’re right that it
should be looking in the resource/ folder for resources, and that’s
what mine’s doing here. I’ve just done a fresh checkout from the
repo and tried it OK, so please can you try pulling, deleting the
build-output directory and try again?
Re 3 all resources being copied: fixed (this should fix your ulimit issue too)
Re 4 UploadMgr: yes that’s exactly how to add a contrib. I’ve fixed
the problem in UploadMgr so that it is not strict mode compatible
and added it to the qxt app as a demo.
Re 5 this.self() not working: fixed
Re 6 environment settings: yes, I’ve added a property to
qxcompiler.Maker that is a map you can set; test/compile-app-demo.js
// Makers use an Analyser to figure out what the Target should write
var maker = new qxcompiler.makers.SimpleApp("qxt.Application",
"qxt.theme.Theme").set({
// Targets know how to output an application
target: new
qxcompiler.targets.SourceTarget("../testdata/qxt/source-output")
});
maker.setEnvironment({
"qxt.customEnvironment": "this is custom (source target)"
});
Re 7: Locales and translation: Locales are supported but
translations not yet; it should be straightforward though, so I’ll
take a look at translations maybe tonight, probably tomorrow
Re 8: Large build-target output: the output isn’t
compressed/minified yet, but I’ll be adding that in “real soon now”;
I’m planning on using UglifyJS to do this, my quick test so far
shows that the size of the qxt app is reduced from 3.9Mb to 1.1Mb,
so not as good yet as generate.py but in the general area. I think
the remaining reduction in size will come by dead code elimination
because the superfluous qx.debug code is not removed from the Build
Target yet
Well I’m pleased with that for a morning’s work :) There’s a bit
more to do that I’ll get onto tonight/tomorrow but it should be
enough to get you going on the next step. Don’t forget to checkout
the qooxdoo repo as well as the main QxCompiler one.
Cheers
John
Date: Monday, 15 February 2016 at 03:28
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster
compilation, and 100% Javascript API to building applications
Hi John! Ready for another dozen of questions and bug reports? ;)
1. As for testdata/qxt directory - yup, it is indeed present, but I
don't see "source" subdirectory, where I would expect the actual
application code to reside. So I just ran "create-application.py -n
qxt" in another location and copied the generated "source"
subdirectory to testdata/qxt. Is it how it is supposed to work?
Function enter: anonymous
Function exit: anonymous
2016-02-15 04:10:22.045 [info ] makers Writing target Build
Target: ../testdata/qxt/build-output/
events.js:141
throw er; // Unhandled 'error' event
^
Error: EMFILE: too many open files, open '../testdata/qxt/build-
output//resource/qx/icon/Tango/48/actions/go-home.png'
at Error (native)
I had to increase open file descriptors limit (ulimit -n 8192 worked
for me). If this cannot be fixed/optimized, probably it should be
reflected in the docs.
3. There seems to be a big mess with resources. Test app works well
in its source variant, but the single-script build fails to load an
GET http://localhost:8282/qxcompiler/testdata/qxt/build-
output/qxt/test.png [HTTP/1.0 404 File not found 2ms]
qxt/test.png boot.js:54698:13
qxt/test.png boot.js:54698:13
000569 ImageLoader: Not recognized format of external image
'qxt/test.png'! boot.js:54698:13
Shouldn't it look in "resources" subdirectory instead? The subdir is
present and does contain resources; however, it seems to contain
*all* the available qooxdoo+app resources (~3600 files, ~25M).
generate.py usually produces a ~150K subdir containing resources
that are actually required.
At the same time, my application was unable to load resources
neither in source nor in build variant. In both cases it tried to
(unsuccessfully) look up resources in that very same "source-output"
or "build-output" directory.
4. Our project uses your (excellent :) UploadMgr contrib. I've added
function(cb) {
maker.addLibrary("/path/to/qooxdoo-contrib/UploadMgr", cb);
},
Is this the right way to integrate a contrib into the project?
Can't yet say whether it worked or not, as I didn't manage to make
SyntaxError: in strict mode code, functions may be declared only at
top level or immediately within another function (source-
output/transpiled//com/zenesis/qx/upload/XhrHandler.js:212:19)
(original file line 212, function sendAsMime definition)
5. One of my classes invokes its static method from a constructor.
this.self(arguments).foo();
The application failed to load because of "this.constructor.self"
this.constructor.foo();
This could potentially break applications that access static members
from constructors and/or instance methods. (I'm not sure if
constructor is some special case.)
6. Is there a way to pass qooxdoo environment keys to the app a la
config.js? Much better if the values could be passed via command
line. Our build system produces metadata (version, build number,
revision, timestamp) which we are planning to pass to the app in the
form of qx.core.Environment keys.
7. What about locales & translation? For us, it is essential because
our app is already bilingual, and more languages are to come.
8. For our app, "generate.py build" produces a ~900K script, and it
takes less than a second for the app to be up and running.
QxCompiler/BuildTarget produces a huge ~5M single script, and
application startup takes up to 10 seconds (!)
What is the recommended way to produce optimized/minified build that
would be comparable to the original one?
Okay, that's enough for today :) Tomorrow I'll try to figure out the
cause of another failure that prevents our app from starting up.
It's something related to JSON unmarshalling; the data is
unmarshalled as auto-generated class
(qx.data.model.prop1"prop2"prop3[123-0]), while it should be an
application class resolved by a marshal delegate. This works fine
with generate.py.
Good luck!
Dimitri
Post by John Spackman
Further to this - I've added some docs about the API, but also
seen that the BuildTarget has a bug and doesn't work - it's only a
minor issue but I probably wont have time to fix it until tonight
John
Sent: Sunday, February 14, 2016 9:50 AM
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster
compilation, and 100% Javascript API to building applications
Hi Dimitri
Thanks for checking out so quickly :)
Re: npm and docs: done
Re: qooxdoo submodule and https: fixed
Re: QOOXDOO_PATH: no - that's a generator.py/config.json setting
and config.json files are completely ignored; when using the
QxCompiler API, once you have told it where the Qooxdoo library is
you have done the equivalent of setting QOOXDOO_PATH. Of course,
as it's API based you can define variables etc as suits you best :)
maker.addLibrary("../qooxdoo/framework", cb);
Just change the path to point to your preferred location for
qooxdoo. To make this a bit clearer, I've just modified the
test/compile-app-demo.js to declare a variable QOOXDOO_PATH
I expect that the command line version will imply have a search
path for libraries and just auto-discover by searching for
Manifest.json files.
Re: missing testdata/qxt: something went wrong with your checkout,
that directory is definitely there.
Re: build (and source-hybrid) targets: to control output for
Source vs Build vs Source-Hybrid, the API classes
qxcompiler.targets.SourceTarget, qxcompiler.targets.BuildTarget,
and qxcompiler.targets.SourceHybridTarget are used; test/compile-
app-demo.js uses the SourceTarget but you should be able to switch
it easily enough. I say "should" because I realise now that I
havn't tested them for a few weeks now and I should give them the
once over ASAP :) I've got to go out shortly so I'll take a look
this evening.
John
Sent: Sunday, February 14, 2016 2:27 AM
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster
compilation, and 100% Javascript API to building applications
John, congratulations with the long awaited release! :) qooxdoo
guys, do you think that project like this could at some moment
land in qooxdoo and get official support? What about qooxdoo
patches (mostly strict mode compatibility related, AFAIK) - could
they be merged upstream? This would reduce maintainship costs for
those who want to experiment with both QxCompiler and official
toolchain. I didn't yet try QxCompiler with my project - I feel I
yet lack some understanding of how it works. However, I tried to
test it with the skeleton app. Off the top of my head: - one needs
to do "npm install" first and to run test scripts with "node
.js". This might be quite obvious for those experienced
with Node, but I guess the docs will nevertheless benefit from
mentioning this; - it's not easy to clone a repo unless you've set
up Github SSH access. This is because of "qooxdoo" submodule
be a HTTPS URL instead? - does QxCompiler honour QOOXDOO_PATH
setting? How do I define qooxdoo location if it is different from
the bundled one? - testdata/qxt directory doesn't contain skeleton
app. In order to play with test scripts, one needs to create the
app manually (as "skeleton.Application"?) and copy it to the said
location; - does lib/compiler.jar really belong there? Most
importantly, I was unable to find clear instructions how to
produce a single-script minified build (a-la "generate.py build").
By deafult, QxCompiler produces a bunch of JS files, which
significantly slows down loading and, obviously, is not acceptable
for production. Is it possible at all with the current version
(maybe in combination with classic generator)? I hope these are
just minor issues. After all, you've done a great job :) I wish
you good luck and further progress wih QxCompiler. Dimitri > Hi
all > > There is a first release of my QxCompiler that adds ES6 to
Qooxdoo > applications and replaces the generate.py toolchain with
a faster, > 100% Javascript tool that is easily extensible. > >
https://github.com/joh> nspackman/qxcompiler > > It’s an alpha
release, but something that’s been in development for a > while
now and which I’m starting to build into my production > servers.
Post by John Spackman
I’m very open to pull requests or collaboration, and keen to
see this > become a useful tool for myself as well as others so
any questions > etc please ask. > > Regards > John > > > > > > ----
--------------------------------------------------------------- > -
---------- > Site24x7 APM Insight: Get Deep Visibility into
Application > Performance > APM + Mobile APM + RUM: Monitor 3 App
instances at just $35/Month > Monitor end-to-end web transactions
and take corrective actions now > Troubleshoot faster and improve
end-user experience. Signup Now! >
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140 >
_______________________________________________ > qooxdoo-devel
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel---------
-------------------------------------------------------------------
-- Site24x7 APM Insight: Get Deep Visibility into Application
Performance APM + Mobile APM + RUM: Monitor 3 App instances at
just $35/Month Monitor end-to-end web transactions and take
corrective actions now Troubleshoot faster and improve end-user
experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________ qooxdoo-devel
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
-------------------------------------------------------------------
-----------
Site24x7 APM Insight: Get Deep Visibility into Application
Performance APM + Mobile APM + RUM: Monitor 3 App instances at
just $35/Month Monitor end-to-end web transactions and take
corrective actions now Troubleshoot faster and improve end-user
experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
---------------------------------------------------------------------
--------- Site24x7 APM Insight: Get Deep Visibility into Application
Performance APM + Mobile APM + RUM: Monitor 3 App instances at just
$35/Month Monitor end-to-end web transactions and take corrective
actions now Troubleshoot faster and improve end-user experience.
Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/
4140_______________________________________________ qooxdoo-devel
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
---------------------------------------------------------------------
---------
Site24x7 APM Insight: Get Deep Visibility into Application
Performance APM + Mobile APM + RUM: Monitor 3 App instances at just
$35/Month Monitor end-to-end web transactions and take corrective
actions now Troubleshoot faster and improve end-user experience.
Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
John Spackman
2016-02-16 07:26:28 UTC
Permalink
Hi Dimitri

So it sounds like resources with paths have an issue, or maybe libraries with “.” in the namespace – I’ll have a look this morning but as it’s happening with UploadMgr then I have a reproducible case I can work on.

I’ve had a look at the stack trace – but I can’t see any clues to what might be causing it; by marshalling, do you mean the qx.data.marshall.*? That should not be affected, the only change to code was to allow .defer() to not be called immediately after the class was defined. I’ve just realised that this means that dynamically defined classes will not have their defer called, and there’s a fix in the next commit for that but I don’t think that’s the issue your coming across.

Re Node 0.12: I think there was another problem with early versions of Node <= 0.12 other than just language – I can’t remember exactly what now but there was something that didn’t work properly until I upgraded. But you’re right, QxCompiler could be babelified :) At one point, the Babel plugin part was written as a proper, standalone babel plugin in ES6 but I had some trouble getting Webstorm to compile and debug properly so I switched back to native code

Re build server: I’d expect QxCompiler to be installed once, globally and then used externally to the projects, i.e. your build code could do the project checkout and then use the API to build each project. The project wouldn’t know about QxCompiler at all.

This sounds similar to my use case, except my builds are on-the-fly on product servers rather than as part of release: In my case I have multi homed Tomcat servers, and each website uses Qooxdoo on the server and client; the server apps generate html (think ASP/PHP/JSP but with javascript & Qooxdoo running under Rhino) as well as implement server processes, and there is often more than one client application per website. Doing a major release requires deleting and recompiling all applications (and typically the cache also), and with a minimum of 2 apps (1 server + 1 client) per website that can be quite expensive. At the moment, making a minor patch to the presentation of the website (e.g. modifying the html output from my PHP/ASP-like pages) means an expensive build process too which is pretty tedious.

So what I want is to have QxCompiler sitting as a background process, watching all these applications and the source files they depend on; when a source file changes it rebuilds the app instantly (e.g. 250ms per affected application) and notifies Tomcat to reload the server code if necessary. At the moment, incremental builds take QxCompiler approx 700ms including the (not insignificant) overhead of starting up node.

Re requirements: I’ve updated the docs to match

I’m working on QxCompiler again this morning and should have an update for you by lunchtime

Cheers
John

From: Dimitri <***@cargosoft.ru>
Reply-To: qooxdoo Development <qooxdoo-***@lists.sourceforge.net>
Date: Monday, 15 February 2016 at 18:16
To: <qooxdoo-***@lists.sourceforge.net>
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster compilation, and 100% Javascript API to building applications

Hi! Tried again - things are much better now :) However, two issues still remain, one of them being resource management.

Indeed, test app loads its icons correctly in both versions. Unfortunately, this is not the case for my app. What I've found yet:

- the app loads all the qx.* resources correctly (window minimize/maximize/close icons etc.);
- none of the application's own resources could be loaded. The request is made to "build-output/myapp/foo.png", which obviously results in a 404.
- in resource-db.json, all resources that *do* load contain non-empty metadata block, while those that do *not* have an empty literal {}. The same could be observed in qxt for com/zenesis/qx/upload/* resources.

(Please note that our app contains not only images, but audio files in different formats, mainly mp3/ogg. We put them into "resource/sounds".)

The app still cannot load. That unmarshalling error seems to have gone, but now I've got a "too much recursion" somewhere around the same location in code. (stacktrace)
I'll try to produce MWEs for both issues later today. Some other stuff to consider:

1. NodeJS <= 0.12 is not supported, while 4.x and 5.x are fine (due to object literal extension used at lib/qxcompiler/ClassFile.js:329). Some distros still ship NodeJS 0.x with their stable versions. I think this should be mentioned in the docs. OTOH, why not babelize QxCompiler itself? :)
2. What practice would you recommend for using QxCompiler on a build server? For each build, a clean checkout is made. Doing npm install in the project directory is very expensive (~15MB deps). Assuming QxCompiler resides in its own directory, what would you recommend? symlink <project>/node_modules -> <QxCompiler>/node_modules? install deps with npm -g and use linklocal ? Ideally, I'd like some practice that wouldn't change when we finally have a NPM distro of QxCompiler.
3. It's worth mentioning in the docs that ImageMagick is required (due to "identify" binary being used in qxcompiler/ResourceManager.js).

Cheers,
Dimitri
Dietrich Streifert
2016-02-16 08:18:13 UTC
Permalink
Hi John,

we are using Centos 7.2 aka RHEL 7.2 which delivers nodejs 0.10.36
through epel repository.

Regards
Dietrich
Post by John Spackman
Re Node 0.12: I think there was another problem with early versions of
Node <= 0.12 other than just language – I can’t remember exactly what
now but there was something that didn’t work properly until I
upgraded. But you’re right, QxCompiler could be babelified :) At one
point, the Babel plugin part was written as a proper, standalone babel
plugin in ES6 but I had some trouble getting Webstorm to compile and
debug properly so I switched back to native code
John Spackman
2016-02-16 14:10:46 UTC
Permalink
Hi Dietrich

Yes, I see your point - I use Centos 6.x so I have the same issue. When the core functionality is settled, I’ll go back and look again at using Babel and see what version of Node we can support

Regards
John
Post by Dietrich Streifert
Hi John,
we are using Centos 7.2 aka RHEL 7.2 which delivers nodejs 0.10.36
through epel repository.
Regards
Dietrich
Post by John Spackman
Re Node 0.12: I think there was another problem with early versions of
Node <= 0.12 other than just language – I can’t remember exactly what
now but there was something that didn’t work properly until I
upgraded. But you’re right, QxCompiler could be babelified :) At one
point, the Babel plugin part was written as a proper, standalone babel
plugin in ES6 but I had some trouble getting Webstorm to compile and
debug properly so I switched back to native code
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
John Spackman
2016-02-16 14:30:27 UTC
Permalink
Hi Dimitri

So further to this – I’ve made a lot of progress with translations today, but I’ve run out of time so rather than release anything half baked I’ll hold off until tomorrow

John

From: John Spackman <***@zenesis.com>
Reply-To: qooxdoo Development <qooxdoo-***@lists.sourceforge.net>
Date: Tuesday, 16 February 2016 at 07:26
To: qooxdoo Development <qooxdoo-***@lists.sourceforge.net>
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster compilation, and 100% Javascript API to building applications

Hi Dimitri

So it sounds like resources with paths have an issue, or maybe libraries with “.” in the namespace – I’ll have a look this morning but as it’s happening with UploadMgr then I have a reproducible case I can work on.

I’ve had a look at the stack trace – but I can’t see any clues to what might be causing it; by marshalling, do you mean the qx.data.marshall.*? That should not be affected, the only change to code was to allow .defer() to not be called immediately after the class was defined. I’ve just realised that this means that dynamically defined classes will not have their defer called, and there’s a fix in the next commit for that but I don’t think that’s the issue your coming across.

Re Node 0.12: I think there was another problem with early versions of Node <= 0.12 other than just language – I can’t remember exactly what now but there was something that didn’t work properly until I upgraded. But you’re right, QxCompiler could be babelified :) At one point, the Babel plugin part was written as a proper, standalone babel plugin in ES6 but I had some trouble getting Webstorm to compile and debug properly so I switched back to native code

Re build server: I’d expect QxCompiler to be installed once, globally and then used externally to the projects, i.e. your build code could do the project checkout and then use the API to build each project. The project wouldn’t know about QxCompiler at all.

This sounds similar to my use case, except my builds are on-the-fly on product servers rather than as part of release: In my case I have multi homed Tomcat servers, and each website uses Qooxdoo on the server and client; the server apps generate html (think ASP/PHP/JSP but with javascript & Qooxdoo running under Rhino) as well as implement server processes, and there is often more than one client application per website. Doing a major release requires deleting and recompiling all applications (and typically the cache also), and with a minimum of 2 apps (1 server + 1 client) per website that can be quite expensive. At the moment, making a minor patch to the presentation of the website (e.g. modifying the html output from my PHP/ASP-like pages) means an expensive build process too which is pretty tedious.

So what I want is to have QxCompiler sitting as a background process, watching all these applications and the source files they depend on; when a source file changes it rebuilds the app instantly (e.g. 250ms per affected application) and notifies Tomcat to reload the server code if necessary. At the moment, incremental builds take QxCompiler approx 700ms including the (not insignificant) overhead of starting up node.

Re requirements: I’ve updated the docs to match

I’m working on QxCompiler again this morning and should have an update for you by lunchtime

Cheers
John

From: Dimitri <***@cargosoft.ru>
Reply-To: qooxdoo Development <qooxdoo-***@lists.sourceforge.net>
Date: Monday, 15 February 2016 at 18:16
To: <qooxdoo-***@lists.sourceforge.net>
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster compilation, and 100% Javascript API to building applications

Hi! Tried again - things are much better now :) However, two issues still remain, one of them being resource management.

Indeed, test app loads its icons correctly in both versions. Unfortunately, this is not the case for my app. What I've found yet:

- the app loads all the qx.* resources correctly (window minimize/maximize/close icons etc.);
- none of the application's own resources could be loaded. The request is made to "build-output/myapp/foo.png", which obviously results in a 404.
- in resource-db.json, all resources that *do* load contain non-empty metadata block, while those that do *not* have an empty literal {}. The same could be observed in qxt for com/zenesis/qx/upload/* resources.

(Please note that our app contains not only images, but audio files in different formats, mainly mp3/ogg. We put them into "resource/sounds".)

The app still cannot load. That unmarshalling error seems to have gone, but now I've got a "too much recursion" somewhere around the same location in code. (stacktrace)
I'll try to produce MWEs for both issues later today. Some other stuff to consider:

1. NodeJS <= 0.12 is not supported, while 4.x and 5.x are fine (due to object literal extension used at lib/qxcompiler/ClassFile.js:329). Some distros still ship NodeJS 0.x with their stable versions. I think this should be mentioned in the docs. OTOH, why not babelize QxCompiler itself? :)
2. What practice would you recommend for using QxCompiler on a build server? For each build, a clean checkout is made. Doing npm install in the project directory is very expensive (~15MB deps). Assuming QxCompiler resides in its own directory, what would you recommend? symlink <project>/node_modules -> <QxCompiler>/node_modules? install deps with npm -g and use linklocal ? Ideally, I'd like some practice that wouldn't change when we finally have a NPM distro of QxCompiler.
3. It's worth mentioning in the docs that ImageMagick is required (due to "identify" binary being used in qxcompiler/ResourceManager.js).

Cheers,
Dimitri

------------------------------------------------------------------------------ Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140_______________________________________________ qooxdoo-devel mailing list qooxdoo-***@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
Dimitri
2016-02-16 22:27:58 UTC
Permalink
John,
Just to clarify - by "translations" I mean the process of integrating
*.po files into the final build, so that tr() functions correctly pick
up locale and produce localized messages. By "translations" I don't
mean the process of extraction of strings from the source and
generating *.po files. I consider the latter to be of much lower
priority, since it's usually done once. Just wanted to make sure that
you don't focus on a low-priority task due to some misunderstanding :)
Dimitri
Post by John Spackman
Hi Dimitri
So further to this – I’ve made a lot of progress with translations
today, but I’ve run out of time so rather than release anything half
baked I’ll hold off until tomorrow
John
Date: Tuesday, 16 February 2016 at 07:26
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster
compilation, and 100% Javascript API to building applications
Hi Dimitri
So it sounds like resources with paths have an issue, or maybe
libraries with “.” in the namespace – I’ll have a look this morning
but as it’s happening with UploadMgr then I have a reproducible case
I can work on.
I’ve had a look at the stack trace – but I can’t see any clues to
what might be causing it; by marshalling, do you mean the
qx.data.marshall.*?  That should not be affected, the only change to
code was to allow .defer() to not be called immediately after the
class was defined.  I’ve just realised that this means that
dynamically defined classes will not have their defer called, and
there’s a fix in the next commit for that but I don’t think that’s
the issue your coming across.
Re Node 0.12: I think there was another problem with early versions
of Node <= 0.12 other than just language – I can’t remember exactly
what now but there was something that didn’t work properly until I
upgraded.  But you’re right, QxCompiler could be babelified :)  At
one point, the Babel plugin part was written as a proper, standalone
babel plugin in ES6 but I had some trouble getting Webstorm to
compile and debug properly so I switched back to native code
Re build server: I’d expect QxCompiler to be installed once, globally
and then used externally to the projects, i.e. your build code could
do the project checkout and then use the API to build each project.
 The project wouldn’t know about QxCompiler at all.
This sounds similar to my use case, except my builds are on-the-fly
on product servers rather than as part of release: In my case I have
multi homed Tomcat servers, and each website uses Qooxdoo on the
server and client; the server apps generate html (think ASP/PHP/JSP
but with javascript & Qooxdoo running under Rhino) as well as
implement server processes, and there is often more than one client
application per website.  Doing a major release requires deleting and
recompiling all applications (and typically the cache also), and with
a minimum of 2 apps (1 server + 1 client) per website that can be
quite expensive.  At the moment, making a minor patch to the
presentation of the website (e.g. modifying the html output from my
PHP/ASP-like pages) means an expensive build process too which is
pretty tedious.  
So what I want is to have QxCompiler sitting as a background process,
watching all these applications and the source files they depend on;
when a source file changes it rebuilds the app instantly (e.g. 250ms
per affected application) and notifies Tomcat to reload the server
code if necessary.  At the moment, incremental builds take QxCompiler
approx 700ms including the (not insignificant) overhead of starting
up node.
Re requirements: I’ve updated the docs to match
I’m working on QxCompiler again this morning and should have an
update for you by lunchtime 
Cheers
John
Date: Monday, 15 February 2016 at 18:16
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster
compilation, and 100% Javascript API to building applications
Hi! Tried again - things are much better now :) However, two issues
still remain, one of them being resource management.
Indeed, test app loads its icons correctly in both versions.
- the app loads all the qx.* resources correctly (window
minimize/maximize/close icons etc.);
- none of the application's own resources could be loaded. The
request is made to "build-output/myapp/foo.png", which obviously
results in a 404.
- in resource-db.json, all resources that *do* load contain non-empty
metadata block, while those that do *not* have an empty literal {}.
The same could be observed in qxt for com/zenesis/qx/upload/*
resources.
(Please note that our app contains not only images, but audio files
in different formats, mainly mp3/ogg. We put them into
"resource/sounds".)
The app still cannot load. That unmarshalling error seems to have
gone, but now I've got a "too much recursion" somewhere around the
same location in code. (stacktrace)
1. NodeJS <= 0.12 is not supported, while 4.x and 5.x are fine (due
to object literal extension used at lib/qxcompiler/ClassFile.js:329).
Some distros still ship NodeJS 0.x with their stable versions. I
think this should be mentioned in the docs. OTOH, why not babelize
QxCompiler itself? :)
2. What practice would you recommend for using QxCompiler on a build
server? For each build, a clean checkout is made. Doing npm install
in the project directory is very expensive (~15MB deps). Assuming
QxCompiler resides in its own directory, what would you recommend?
symlink /node_modules -> /node_modules? install
deps with npm -g and use linklocal ? Ideally, I'd like some practice
that wouldn't change when we finally have a NPM distro of QxCompiler.
3. It's worth mentioning in the docs that ImageMagick is required
(due to "identify" binary being used in
qxcompiler/ResourceManager.js).
Cheers,
Dimitri
-------------------------------------------------------------------
----------- Site24x7 APM Insight: Get Deep Visibility into
Application Performance APM + Mobile APM + RUM: Monitor 3 App
instances at just $35/Month Monitor end-to-end web transactions and
take corrective actions now Troubleshoot faster and improve end-user
experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id
=272487151&iu=/4140_______________________________________________
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
-------------------------------------------------------------------
-----------
Site24x7 APM Insight: Get Deep Visibility into Application
Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
Dimitri
2016-02-16 21:34:52 UTC
Permalink
Hi John,
Sorry for all the stir I've raised around resource handling. The
problem was in our code. It turned out that we were still using old-
style compiler hints (#asset, #use, #require etc.) ignored by
QxCompiler.
This in fact caused both issues (resources and unmarshalling). With
#asset being ignored, obviously, no resources could be loaded. The
second case was a bit more complex. By some obscure reason (Java
habit?), we used the following construct in IMarshalDelegate
implementation:
/**
 #require(foo.Bar)
*/
...
    getModelClass: function(properties) {
        switch(properties) {
            case 'foo"bar':
                return qx.Class.getByName("foo.Bar");
            default:
                return null;
        }
    }
Obviously, foo.Bar wasn't available at runtime, and the data was
unmarshalled to default (generated) class, while foo.Bar was expected.
I've eliminated compiler hints completely and replaced
qx.Class.getByName calls with direct references. Everything works
perfectly now! :)
(I don't think old-style syntax is worth being supported in QxCompiler,
but in order to aid oblivious guys like me it would be nice to mention
in the docs that #-hints are not supported and should be replaced with
@-hints.)
The only two remaining issues are translation (the app runs with its
default locale and doesn't pick up *.po files) and script size for
single-script build. Load time is much better now, I guess that 10
second delay was caused by unoptimized resource handling. Yet I didn't
try any actual Babel transformations; as far as I know, it doesn't do
anything by default. For those unfamiliar with Babel, it would be nice
if the docs contained some examples of how to actually enable Babel
transformations.
As for build setup, I've come up with the following. The script that
drives the build lives in VCS in the project directory; upon checkout,
"npm link async" is executed, which is a cheap operation that installs
a symlink for async module (previously installed globally). It's the
only module that is required by the driver script; the rest is
successfully pulled from QxCompiler directory.
Cheers, and keep going! QxCompiler is definitely the coolest thing that
happened to qooxdoo in last months and maybe years :)
Dimitri
Post by John Spackman
Hi Dimitri
So it sounds like resources with paths have an issue, or maybe
libraries with “.” in the namespace – I’ll have a look this morning
but as it’s happening with UploadMgr then I have a reproducible case
I can work on.
I’ve had a look at the stack trace – but I can’t see any clues to
what might be causing it; by marshalling, do you mean the
qx.data.marshall.*?  That should not be affected, the only change to
code was to allow .defer() to not be called immediately after the
class was defined.  I’ve just realised that this means that
dynamically defined classes will not have their defer called, and
there’s a fix in the next commit for that but I don’t think that’s
the issue your coming across.
Re Node 0.12: I think there was another problem with early versions
of Node <= 0.12 other than just language – I can’t remember exactly
what now but there was something that didn’t work properly until I
upgraded.  But you’re right, QxCompiler could be babelified :)  At
one point, the Babel plugin part was written as a proper, standalone
babel plugin in ES6 but I had some trouble getting Webstorm to
compile and debug properly so I switched back to native code
Re build server: I’d expect QxCompiler to be installed once, globally
and then used externally to the projects, i.e. your build code could
do the project checkout and then use the API to build each project.
 The project wouldn’t know about QxCompiler at all.
This sounds similar to my use case, except my builds are on-the-fly
on product servers rather than as part of release: In my case I have
multi homed Tomcat servers, and each website uses Qooxdoo on the
server and client; the server apps generate html (think ASP/PHP/JSP
but with javascript & Qooxdoo running under Rhino) as well as
implement server processes, and there is often more than one client
application per website.  Doing a major release requires deleting and
recompiling all applications (and typically the cache also), and with
a minimum of 2 apps (1 server + 1 client) per website that can be
quite expensive.  At the moment, making a minor patch to the
presentation of the website (e.g. modifying the html output from my
PHP/ASP-like pages) means an expensive build process too which is
pretty tedious.  
So what I want is to have QxCompiler sitting as a background process,
watching all these applications and the source files they depend on;
when a source file changes it rebuilds the app instantly (e.g. 250ms
per affected application) and notifies Tomcat to reload the server
code if necessary.  At the moment, incremental builds take QxCompiler
approx 700ms including the (not insignificant) overhead of starting
up node.
Re requirements: I’ve updated the docs to match
I’m working on QxCompiler again this morning and should have an
update for you by lunchtime 
Cheers
John
John Spackman
2016-02-17 09:08:35 UTC
Permalink
Hi Dimitri

No worries, those are the kind of bug reports I like to hear, ones where it’s not my fault! :D

Actually I’m a little surprised that the generator supports the # compiler hints still, I thought that had been dropped ages ago; I have yet to add warnings output for unresolved symbols etc so I’ll include warnings for old style hints too.

Babel is on by default, so there’s nothing extra you have to do to start using ES6. In the qxt Application.js there is this:
button1.addListener("execute", () => alert("Hello World!"));
I’ve got some customer work I have to get done this morning but hopefully have some time put aside this afternoon to finish off translations. It’s not too big a deal, most of the work is in reading and writing the .po file format so both the extraction of strings process and the loading of translations will be included. Hopefully todays release will include build target compression too.

Regards
John

From: Dimitri <***@cargosoft.ru>
Reply-To: qooxdoo Development <qooxdoo-***@lists.sourceforge.net>
Date: Tuesday, 16 February 2016 at 21:34
To: <qooxdoo-***@lists.sourceforge.net>
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster compilation, and 100% Javascript API to building applications

Hi John,

Sorry for all the stir I've raised around resource handling. The problem was in our code. It turned out that we were still using old-style compiler hints (#asset, #use, #require etc.) ignored by QxCompiler.

This in fact caused both issues (resources and unmarshalling). With #asset being ignored, obviously, no resources could be loaded. The second case was a bit more complex. By some obscure reason (Java habit?), we used the following construct in IMarshalDelegate implementation:

/**
#require(foo.Bar)
*/
...
getModelClass: function(properties) {
switch(properties) {
case 'foo"bar':
return qx.Class.getByName("foo.Bar");
default:
return null;
}
}
Obviously, foo.Bar wasn't available at runtime, and the data was unmarshalled to default (generated) class, while foo.Bar was expected. I've eliminated compiler hints completely and replaced qx.Class.getByName calls with direct references. Everything works perfectly now! :)

(I don't think old-style syntax is worth being supported in QxCompiler, but in order to aid oblivious guys like me it would be nice to mention in the docs that #-hints are not supported and should be replaced with @-hints.)

The only two remaining issues are translation (the app runs with its default locale and doesn't pick up *.po files) and script size for single-script build. Load time is much better now, I guess that 10 second delay was caused by unoptimized resource handling. Yet I didn't try any actual Babel transformations; as far as I know, it doesn't do anything by default. For those unfamiliar with Babel, it would be nice if the docs contained some examples of how to actually enable Babel transformations.

As for build setup, I've come up with the following. The script that drives the build lives in VCS in the project directory; upon checkout, "npm link async" is executed, which is a cheap operation that installs a symlink for async module (previously installed globally). It's the only module that is required by the driver script; the rest is successfully pulled from QxCompiler directory.

Cheers, and keep going! QxCompiler is definitely the coolest thing that happened to qooxdoo in last months and maybe years :)

Dimitri

Hi Dimitri

So it sounds like resources with paths have an issue, or maybe libraries with “.” in the namespace – I’ll have a look this morning but as it’s happening with UploadMgr then I have a reproducible case I can work on.

I’ve had a look at the stack trace – but I can’t see any clues to what might be causing it; by marshalling, do you mean the qx.data.marshall.*? That should not be affected, the only change to code was to allow .defer() to not be called immediately after the class was defined. I’ve just realised that this means that dynamically defined classes will not have their defer called, and there’s a fix in the next commit for that but I don’t think that’s the issue your coming across.

Re Node 0.12: I think there was another problem with early versions of Node <= 0.12 other than just language – I can’t remember exactly what now but there was something that didn’t work properly until I upgraded. But you’re right, QxCompiler could be babelified :) At one point, the Babel plugin part was written as a proper, standalone babel plugin in ES6 but I had some trouble getting Webstorm to compile and debug properly so I switched back to native code

Re build server: I’d expect QxCompiler to be installed once, globally and then used externally to the projects, i.e. your build code could do the project checkout and then use the API to build each project. The project wouldn’t know about QxCompiler at all.

This sounds similar to my use case, except my builds are on-the-fly on product servers rather than as part of release: In my case I have multi homed Tomcat servers, and each website uses Qooxdoo on the server and client; the server apps generate html (think ASP/PHP/JSP but with javascript & Qooxdoo running under Rhino) as well as implement server processes, and there is often more than one client application per website. Doing a major release requires deleting and recompiling all applications (and typically the cache also), and with a minimum of 2 apps (1 server + 1 client) per website that can be quite expensive. At the moment, making a minor patch to the presentation of the website (e.g. modifying the html output from my PHP/ASP-like pages) means an expensive build process too which is pretty tedious.

So what I want is to have QxCompiler sitting as a background process, watching all these applications and the source files they depend on; when a source file changes it rebuilds the app instantly (e.g. 250ms per affected application) and notifies Tomcat to reload the server code if necessary. At the moment, incremental builds take QxCompiler approx 700ms including the (not insignificant) overhead of starting up node.

Re requirements: I’ve updated the docs to match

I’m working on QxCompiler again this morning and should have an update for you by lunchtime

Cheers
John

------------------------------------------------------------------------------ Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140_______________________________________________ qooxdoo-devel mailing list qooxdoo-***@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
John Spackman
2016-02-17 17:56:11 UTC
Permalink
Hi again

There’s a new release which includes support for translations; you don’t have to do anything to enable it, just rerun the qxcompiler.

If you want to update your .po files with new translation strings, there is an example of how to do this for the qxt application in test/update-app-translations-demo.js, but basically after maker.make() has completed, you just call maker.updateTranslations()
// ...snip...
/*
* Make it
*/
function (cb) {
maker.make(cb);
},

/*
* Translations
*/
function (cb) {
maker.updateTranslations(appLibrary, cb);
}
// ...snip...
BUT:: because this modifies the original *.po files in your source directory, please take a backup of your files first.

Build Target compression is next :)

John

From: John Spackman <***@zenesis.com>
Reply-To: qooxdoo Development <qooxdoo-***@lists.sourceforge.net>
Date: Wednesday, 17 February 2016 at 09:08
To: qooxdoo Development <qooxdoo-***@lists.sourceforge.net>
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster compilation, and 100% Javascript API to building applications

Hi Dimitri

No worries, those are the kind of bug reports I like to hear, ones where it’s not my fault! :D

Actually I’m a little surprised that the generator supports the # compiler hints still, I thought that had been dropped ages ago; I have yet to add warnings output for unresolved symbols etc so I’ll include warnings for old style hints too.

Babel is on by default, so there’s nothing extra you have to do to start using ES6. In the qxt Application.js there is this:
button1.addListener("execute", () => alert("Hello World!"));
I’ve got some customer work I have to get done this morning but hopefully have some time put aside this afternoon to finish off translations. It’s not too big a deal, most of the work is in reading and writing the .po file format so both the extraction of strings process and the loading of translations will be included. Hopefully todays release will include build target compression too.

Regards
John

From: Dimitri <***@cargosoft.ru>
Reply-To: qooxdoo Development <qooxdoo-***@lists.sourceforge.net>
Date: Tuesday, 16 February 2016 at 21:34
To: <qooxdoo-***@lists.sourceforge.net>
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster compilation, and 100% Javascript API to building applications

Hi John,

Sorry for all the stir I've raised around resource handling. The problem was in our code. It turned out that we were still using old-style compiler hints (#asset, #use, #require etc.) ignored by QxCompiler.

This in fact caused both issues (resources and unmarshalling). With #asset being ignored, obviously, no resources could be loaded. The second case was a bit more complex. By some obscure reason (Java habit?), we used the following construct in IMarshalDelegate implementation:

/**
#require(foo.Bar)
*/
...
getModelClass: function(properties) {
switch(properties) {
case 'foo"bar':
return qx.Class.getByName("foo.Bar");
default:
return null;
}
}
Obviously, foo.Bar wasn't available at runtime, and the data was unmarshalled to default (generated) class, while foo.Bar was expected. I've eliminated compiler hints completely and replaced qx.Class.getByName calls with direct references. Everything works perfectly now! :)

(I don't think old-style syntax is worth being supported in QxCompiler, but in order to aid oblivious guys like me it would be nice to mention in the docs that #-hints are not supported and should be replaced with @-hints.)

The only two remaining issues are translation (the app runs with its default locale and doesn't pick up *.po files) and script size for single-script build. Load time is much better now, I guess that 10 second delay was caused by unoptimized resource handling. Yet I didn't try any actual Babel transformations; as far as I know, it doesn't do anything by default. For those unfamiliar with Babel, it would be nice if the docs contained some examples of how to actually enable Babel transformations.

As for build setup, I've come up with the following. The script that drives the build lives in VCS in the project directory; upon checkout, "npm link async" is executed, which is a cheap operation that installs a symlink for async module (previously installed globally). It's the only module that is required by the driver script; the rest is successfully pulled from QxCompiler directory.

Cheers, and keep going! QxCompiler is definitely the coolest thing that happened to qooxdoo in last months and maybe years :)

Dimitri

Hi Dimitri

So it sounds like resources with paths have an issue, or maybe libraries with “.” in the namespace – I’ll have a look this morning but as it’s happening with UploadMgr then I have a reproducible case I can work on.

I’ve had a look at the stack trace – but I can’t see any clues to what might be causing it; by marshalling, do you mean the qx.data.marshall.*? That should not be affected, the only change to code was to allow .defer() to not be called immediately after the class was defined. I’ve just realised that this means that dynamically defined classes will not have their defer called, and there’s a fix in the next commit for that but I don’t think that’s the issue your coming across.

Re Node 0.12: I think there was another problem with early versions of Node <= 0.12 other than just language – I can’t remember exactly what now but there was something that didn’t work properly until I upgraded. But you’re right, QxCompiler could be babelified :) At one point, the Babel plugin part was written as a proper, standalone babel plugin in ES6 but I had some trouble getting Webstorm to compile and debug properly so I switched back to native code

Re build server: I’d expect QxCompiler to be installed once, globally and then used externally to the projects, i.e. your build code could do the project checkout and then use the API to build each project. The project wouldn’t know about QxCompiler at all.

This sounds similar to my use case, except my builds are on-the-fly on product servers rather than as part of release: In my case I have multi homed Tomcat servers, and each website uses Qooxdoo on the server and client; the server apps generate html (think ASP/PHP/JSP but with javascript & Qooxdoo running under Rhino) as well as implement server processes, and there is often more than one client application per website. Doing a major release requires deleting and recompiling all applications (and typically the cache also), and with a minimum of 2 apps (1 server + 1 client) per website that can be quite expensive. At the moment, making a minor patch to the presentation of the website (e.g. modifying the html output from my PHP/ASP-like pages) means an expensive build process too which is pretty tedious.

So what I want is to have QxCompiler sitting as a background process, watching all these applications and the source files they depend on; when a source file changes it rebuilds the app instantly (e.g. 250ms per affected application) and notifies Tomcat to reload the server code if necessary. At the moment, incremental builds take QxCompiler approx 700ms including the (not insignificant) overhead of starting up node.

Re requirements: I’ve updated the docs to match

I’m working on QxCompiler again this morning and should have an update for you by lunchtime

Cheers
John

------------------------------------------------------------------------------ Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140_______________________________________________ qooxdoo-devel mailing list qooxdoo-***@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
------------------------------------------------------------------------------ Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140_______________________________________________ qooxdoo-devel mailing list qooxdoo-***@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
Dimitri
2016-02-17 18:44:46 UTC
Permalink
Hi John,
That pace you're keeping, it's incredible :) Glad to hear translations
finally landed!
Unfortunately, I couldn't make them work :(
[***@localhost test]$ node compile-app-demo.js
[BABEL] Note: The code generator has deoptimised the styling of
"/home/mitya/qxcompiler/qooxdoo/framework/source/class/qx/ui/core/Widge
t.js" as it exceeds the max of "100KB".
2016-02-17 21:30:13.229 [info ] makers Writing target Source
Target: ../testdata/qxt/source-output/
/home/mitya/qxcompiler/lib/qxcompiler/targets/Target.js:239
pkgdata.locales["C"] = db.cldr["en"];
^

TypeError: Cannot read property 'en' of undefined
at /home/mitya/qxcompiler/lib/qxcompiler/targets/Target.js:239:45
at /home/mitya/qxcompiler/node_modules/async/lib/async.js:718:13
at async.forEachOf.async.eachOf
(/home/mitya/qxcompiler/node_modules/async/lib/async.js:233:13)
at _parallel
(/home/mitya/qxcompiler/node_modules/async/lib/async.js:717:9)
at Object.async.parallel
(/home/mitya/qxcompiler/node_modules/async/lib/async.js:731:9)
at /home/mitya/qxcompiler/lib/qxcompiler/targets/Target.js:236:15
at /home/mitya/qxcompiler/lib/util.js:321:9
at /home/mitya/qxcompiler/node_modules/async/lib/async.js:52:16
at /home/mitya/qxcompiler/node_modules/async/lib/async.js:269:32
at /home/mitya/qxcompiler/node_modules/async/lib/async.js:44:16
(removed qxcompiler dir & did clean checkout, didn't help either)
Re: Babel - sadly, didn't make it work either :( that arrow function in
qxt/Application.js goes unchanged first to [source|build]-
output/transpiled, then to [source|build]-output/script. The same with
string templates, etc. Note the above [BABEL] message - that means,
Babel is invoked, but doesn't apply any actual transformations. I tried
"strace -e trace=open node compile-app-demo.js", and didn't see it open
.babelrc or any plugin and/or preset modules. Could you please make
sure it's indeed enabled? If it works for you, could you please assist
me in tracing/debugging (since I'm a newbie in a Node world)?
As for script compression, I played a bit with the standalone uglify tool. Works pretty fine, but it takes ~30 seconds to process a skeleton app. Do you think it can be sped up if integrated into QxCompiler?
Minor issue: test/*.js scripts could have a shebang string (#!/usr/bin/node) and +x mode so that they could be run directly, without typing "node ..." each time.
Cheers!
Dimitri
Post by John Spackman
Hi again
There’s a new release which includes support for translations; you
don’t have to do anything to enable it, just rerun the qxcompiler.
If you want to update your .po files with new translation strings,
there is an example of how to do this for the qxt application in
test/update-app-translations-demo.js, but basically after
maker.make() has completed, you just call maker.updateTranslations()
// ...snip...
/*
 * Make it
 */
function (cb) {
  maker.make(cb);
},
/*
 * Translations
 */
function (cb) {
  maker.updateTranslations(appLibrary, cb);
}
// ...snip...
BUT:: because this modifies the original *.po files in your source
directory, please take a backup of your files first.
Build Target compression is next :)
John
Date: Wednesday, 17 February 2016 at 09:08
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster
compilation, and 100% Javascript API to building applications
Hi Dimitri
No worries, those are the kind of bug reports I like to hear, ones
where it’s not my fault! :D
Actually I’m a little surprised that the generator supports the #
compiler hints still, I thought that had been dropped ages ago; I
have yet to add warnings output for unresolved symbols etc so I’ll
include warnings for old style hints too.
Babel is on by default, so there’s nothing extra you have to do to
button1.addListener("execute", () => alert("Hello World!"));
I’ve got some customer work I have to get done this morning but
hopefully have some time put aside this afternoon to finish off
translations.  It’s not too big a deal, most of the work is in
reading and writing the .po file format so both the extraction of
strings process and the loading of translations will be included.
 Hopefully todays release will include build target compression too.
Regards
John
Date: Tuesday, 16 February 2016 at 21:34
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster
compilation, and 100% Javascript API to building applications
Hi John,
Sorry for all the stir I've raised around resource handling. The
problem was in our code. It turned out that we were still using old-
style compiler hints (#asset, #use, #require etc.) ignored by
QxCompiler.
This in fact caused both issues (resources and unmarshalling). With
#asset being ignored, obviously, no resources could be loaded. The
second case was a bit more complex. By some obscure reason (Java
habit?), we used the following construct in IMarshalDelegate
/**
 #require(foo.Bar)
*/
...
    getModelClass: function(properties) {
        switch(properties) {
                return qx.Class.getByName("foo.Bar");
                return null;
        }
    }
Obviously, foo.Bar wasn't available at runtime, and the data was
unmarshalled to default (generated) class, while foo.Bar was
expected. I've eliminated compiler hints completely and replaced
qx.Class.getByName calls with direct references. Everything works
perfectly now! :)
(I don't think old-style syntax is worth being supported in
QxCompiler, but in order to aid oblivious guys like me it would be
nice to mention in the docs that #-hints are not supported and should
The only two remaining issues are translation (the app runs with its
default locale and doesn't pick up *.po files) and script size for
single-script build. Load time is much better now, I guess that 10
second delay was caused by unoptimized resource handling. Yet I
didn't try any actual Babel transformations; as far as I know, it
doesn't do anything by default. For those unfamiliar with Babel, it
would be nice if the docs contained some examples of how to actually
enable Babel transformations.
As for build setup, I've come up with the following. The script that
drives the build lives in VCS in the project directory; upon
checkout, "npm link async" is executed, which is a cheap operation
that installs a symlink for async module (previously installed
globally). It's the only module that is required by the driver
script; the rest is successfully pulled from QxCompiler directory.
Cheers, and keep going! QxCompiler is definitely the coolest thing
that happened to qooxdoo in last months and maybe years :)
Dimitri
Post by John Spackman
Hi Dimitri
So it sounds like resources with paths have an issue, or maybe
libraries with “.” in the namespace – I’ll have a look this morning
but as it’s happening with UploadMgr then I have a reproducible
case I can work on.
I’ve had a look at the stack trace – but I can’t see any clues to
what might be causing it; by marshalling, do you mean the
qx.data.marshall.*?  That should not be affected, the only change
to code was to allow .defer() to not be called immediately after
the class was defined.  I’ve just realised that this means that
dynamically defined classes will not have their defer called, and
there’s a fix in the next commit for that but I don’t think that’s
the issue your coming across.
Re Node 0.12: I think there was another problem with early versions
of Node <= 0.12 other than just language – I can’t remember exactly
what now but there was something that didn’t work properly until I
upgraded.  But you’re right, QxCompiler could be babelified :)  At
one point, the Babel plugin part was written as a proper,
standalone babel plugin in ES6 but I had some trouble getting
Webstorm to compile and debug properly so I switched back to native
code
Re build server: I’d expect QxCompiler to be installed once,
globally and then used externally to the projects, i.e. your build
code could do the project checkout and then use the API to build
each project.  The project wouldn’t know about QxCompiler at all.
This sounds similar to my use case, except my builds are on-the-fly
on product servers rather than as part of release: In my case I
have multi homed Tomcat servers, and each website uses Qooxdoo on
the server and client; the server apps generate html (think
ASP/PHP/JSP but with javascript & Qooxdoo running under Rhino) as
well as implement server processes, and there is often more than
one client application per website.  Doing a major release requires
deleting and recompiling all applications (and typically the cache
also), and with a minimum of 2 apps (1 server + 1 client) per
website that can be quite expensive.  At the moment, making a minor
patch to the presentation of the website (e.g. modifying the html
output from my PHP/ASP-like pages) means an expensive build process
too which is pretty tedious.  
So what I want is to have QxCompiler sitting as a background
process, watching all these applications and the source files they
depend on; when a source file changes it rebuilds the app instantly
(e.g. 250ms per affected application) and notifies Tomcat to reload
the server code if necessary.  At the moment, incremental builds
take QxCompiler approx 700ms including the (not insignificant)
overhead of starting up node.
Re requirements: I’ve updated the docs to match
I’m working on QxCompiler again this morning and should have an
update for you by lunchtime 
Cheers
John
-------------------------------------------------------------------
----------- Site24x7 APM Insight: Get Deep Visibility into
Application Performance APM + Mobile APM + RUM: Monitor 3 App
instances at just $35/Month Monitor end-to-end web transactions and
take corrective actions now Troubleshoot faster and improve end-user
experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id
=272487151&iu=/4140_______________________________________________
qooxdoo-devel mailing list qooxdoo-
fo/qooxdoo-devel
-------------------------------------------------------------------
----------- Site24x7 APM Insight: Get Deep Visibility into
Application Performance APM + Mobile APM + RUM: Monitor 3 App
instances at just $35/Month Monitor end-to-end web transactions and
take corrective actions now Troubleshoot faster and improve end-user
experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id
=272487151&iu=/4140_______________________________________________
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
-------------------------------------------------------------------
-----------
Site24x7 APM Insight: Get Deep Visibility into Application
Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
Dimitri
2016-02-17 19:33:23 UTC
Permalink
Just a quick follow-up on Babel - I've rolled back to yesterday's
version, edited lib/qxcompiler/ClassFile.js:218, replaced that plugins
array with presets: [ "es2015" ]. Suddenly, it worked :)
I'm still wondering why it doesn't try to look for any .babelrc file.
It's clearly stated in the docs that the "babelrc" option is true by
default. And it definitely would be good to have control over Babel via
(optional) config file.
Another issue to consider is that not every ES6 feature is implemented
via code transformation. Some (ex., Promises) require babel-polyfill.
(Mention in the docs? Integrate polyfill into build?)
Dimitri
Post by Dietrich Streifert
Hi John,
That pace you're keeping, it's incredible :) Glad to hear
translations finally landed!
Unfortunately, I couldn't make them work :(
[BABEL] Note: The code generator has deoptimised the styling of
"/home/mitya/qxcompiler/qooxdoo/framework/source/class/qx/ui/core/Wid
get.js" as it exceeds the max of "100KB".
2016-02-17 21:30:13.229 [info ] makers          Writing target Source
Target: ../testdata/qxt/source-output/
/home/mitya/qxcompiler/lib/qxcompiler/targets/Target.js:239
              pkgdata.locales["C"] = db.cldr["en"];
                                            ^
TypeError: Cannot read property 'en' of undefined
    at /home/mitya/qxcompiler/lib/qxcompiler/targets/Target.js:239:45
    at /home/mitya/qxcompiler/node_modules/async/lib/async.js:718:13
    at async.forEachOf.async.eachOf
(/home/mitya/qxcompiler/node_modules/async/lib/async.js:233:13)
    at _parallel
(/home/mitya/qxcompiler/node_modules/async/lib/async.js:717:9)
    at Object.async.parallel
(/home/mitya/qxcompiler/node_modules/async/lib/async.js:731:9)
    at /home/mitya/qxcompiler/lib/qxcompiler/targets/Target.js:236:15
    at /home/mitya/qxcompiler/lib/util.js:321:9
    at /home/mitya/qxcompiler/node_modules/async/lib/async.js:52:16
    at /home/mitya/qxcompiler/node_modules/async/lib/async.js:269:32
    at /home/mitya/qxcompiler/node_modules/async/lib/async.js:44:16
(removed qxcompiler dir & did clean checkout, didn't help either)
Re: Babel - sadly, didn't make it work either :( that arrow function
in qxt/Application.js goes unchanged first to [source|build]-
output/transpiled, then to [source|build]-output/script. The same
with string templates, etc. Note the above [BABEL] message - that
means, Babel is invoked, but doesn't apply any actual
transformations. I tried "strace -e trace=open node compile-app-
demo.js", and didn't see it open .babelrc or any plugin and/or preset
modules. Could you please make sure it's indeed enabled? If it works
for you, could you please assist me in tracing/debugging (since I'm a
newbie in a Node world)?
As for script compression, I played a bit with the standalone uglify
tool. Works pretty fine, but it takes ~30 seconds to process a
skeleton app. Do you think it can be sped up if integrated into
QxCompiler?
Minor issue: test/*.js scripts could have a shebang string
(#!/usr/bin/node) and +x mode so that they could be run directly,
without typing "node ..." each time.
Cheers!
Dimitri
John Spackman
2016-02-17 21:23:09 UTC
Permalink
Couldn’t resist one more push – I’ve just uploaded another version, this time with uglify for the BuildTarget.

There is a little more work to do here, because there is code that can be eliminated for known qx.core.Environment values, e.g. “qx.debug” code should be stripped; this will reduce the final code size as well as reduce dependencies, which will reduce the code size even more. My guess is that this will bring qxcompiler into the same ball park as generate.py in terms of the size of the final output

John

From: John Spackman <***@zenesis.com>
Reply-To: qooxdoo Development <qooxdoo-***@lists.sourceforge.net>
Date: Wednesday, 17 February 2016 at 20:56
To: qooxdoo Development <qooxdoo-***@lists.sourceforge.net>
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster compilation, and 100% Javascript API to building applications

Got it – it was a bug in my plugin I introduced a while ago, but Chrome obviously supports arrow functions natively so I didn’t pick up that the translation wasn’t happening any more. The .babelrc is not needed for this because we’re using it embedded, I think the .babelrc is only for the CLI

That’s very a good point about the polypill BTW because it’s not being included and it should be, I’ll add that in tomorrow.

The problem with translations is fixed now too, sorry about that I should have deleted the output directory before retesting :oops:

What options are you running Uglify with, and what version are you running? I’m using "uglify boot.js -m -c”, I just timed it on mine and it’s mangled the qxt app in 11 seconds (I’m using uglify 2.6.1) Still not great but better than 30 secs :) I’ve nearly got Uglify done, but stopped for dinner and then saw your email so switched back.

I’ve done the #! entries too

I’ve pushed a new release, give it a go :)

John

From: Dimitri <***@cargosoft.ru>
Reply-To: qooxdoo Development <qooxdoo-***@lists.sourceforge.net>
Date: Wednesday, 17 February 2016 at 19:33
To: <qooxdoo-***@lists.sourceforge.net>
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster compilation, and 100% Javascript API to building applications

Just a quick follow-up on Babel - I've rolled back to yesterday's version, edited lib/qxcompiler/ClassFile.js:218, replaced that plugins array with presets: [ "es2015" ]. Suddenly, it worked :)
I'm still wondering why it doesn't try to look for any .babelrc file. It's clearly stated in the docs that the "babelrc" option is true by default. And it definitely would be good to have control over Babel via (optional) config file.

Another issue to consider is that not every ES6 feature is implemented via code transformation. Some (ex., Promises) require babel-polyfill. (Mention in the docs? Integrate polyfill into build?)

Dimitri

Hi John,

That pace you're keeping, it's incredible :) Glad to hear translations finally landed!

Unfortunately, I couldn't make them work :(

[***@localhost test]$ node compile-app-demo.js
[BABEL] Note: The code generator has deoptimised the styling of "/home/mitya/qxcompiler/qooxdoo/framework/source/class/qx/ui/core/Widget.js" as it exceeds the max of "100KB".
2016-02-17 21:30:13.229 [info ] makers Writing target Source Target: ../testdata/qxt/source-output/
/home/mitya/qxcompiler/lib/qxcompiler/targets/Target.js:239
pkgdata.locales["C"] = db.cldr["en"];
^

TypeError: Cannot read property 'en' of undefined
at /home/mitya/qxcompiler/lib/qxcompiler/targets/Target.js:239:45
at /home/mitya/qxcompiler/node_modules/async/lib/async.js:718:13
at async.forEachOf.async.eachOf (/home/mitya/qxcompiler/node_modules/async/lib/async.js:233:13)
at _parallel (/home/mitya/qxcompiler/node_modules/async/lib/async.js:717:9)
at Object.async.parallel (/home/mitya/qxcompiler/node_modules/async/lib/async.js:731:9)
at /home/mitya/qxcompiler/lib/qxcompiler/targets/Target.js:236:15
at /home/mitya/qxcompiler/lib/util.js:321:9
at /home/mitya/qxcompiler/node_modules/async/lib/async.js:52:16
at /home/mitya/qxcompiler/node_modules/async/lib/async.js:269:32
at /home/mitya/qxcompiler/node_modules/async/lib/async.js:44:16

(removed qxcompiler dir & did clean checkout, didn't help either)

Re: Babel - sadly, didn't make it work either :( that arrow function in qxt/Application.js goes unchanged first to [source|build]-output/transpiled, then to [source|build]-output/script. The same with string templates, etc. Note the above [BABEL] message - that means, Babel is invoked, but doesn't apply any actual transformations. I tried "strace -e trace=open node compile-app-demo.js", and didn't see it open .babelrc or any plugin and/or preset modules. Could you please make sure it's indeed enabled? If it works for you, could you please assist me in tracing/debugging (since I'm a newbie in a Node world)?

As for script compression, I played a bit with the standalone uglify tool. Works pretty fine, but it takes ~30 seconds to process a skeleton app. Do you think it can be sped up if integrated into QxCompiler?

Minor issue: test/*.js scripts could have a shebang string (#!/usr/bin/node) and +x mode so that they could be run directly, without typing "node ..." each time.

Cheers!
Dimitri
------------------------------------------------------------------------------ Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140_______________________________________________ qooxdoo-devel mailing list qooxdoo-***@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
------------------------------------------------------------------------------ Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140_______________________________________________ qooxdoo-devel mailing list qooxdoo-***@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
Dimitri
2016-02-17 22:31:34 UTC
Permalink
No cigar this time :-\
[***@localhost test]$ ./compile-app-demo.js
/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/options/option-manager.js:393
throw new Error("Couldn't find preset " + JSON.stringify(val)
+ " relative to directory " + JSON.stringify(dirname));
^

Error: Couldn't find preset "es2015" relative to directory
"/home/mitya"
at /home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/options/option-manager.js:393:17
at Array.map (native)
at OptionManager.resolvePresets
(/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/options/option-manager.js:385:20)
at OptionManager.mergePresets
(/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/options/option-manager.js:369:10)
at OptionManager.mergeOptions
(/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/options/option-manager.js:328:14)
at OptionManager.addConfig
(/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/options/option-manager.js:234:10)
at OptionManager.findConfigs
(/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/options/option-manager.js:434:16)
at OptionManager.init (/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/options/option-manager.js:482:12)
at File.initOptions (/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/index.js:211:75)
at new File (/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/index.js:129:22)Dunno why it looks for presets in my home dir. In my previous experiment, everything went smooth (except that it didn't in fact collect any qooxdoo deps, since I removed all plugins - but transpiling itself went fine).
As for uglify - ah, that was stupid. I used original uglify, not uglify-js. Just tried the latter (2.6.1) - indeed, it takes ~10 secs for qxt. Not bad.
Post by John Spackman
Got it – it was a bug in my plugin I introduced a while ago, but
Chrome obviously supports arrow functions natively so I didn’t pick
up that the translation wasn’t happening any more.  The .babelrc is
not needed for this because we’re using it embedded, I think the
.babelrc is only for the CLI
That’s very a good point about the polypill BTW because it’s not
being included and it should be, I’ll add that in tomorrow.
The problem with translations is fixed now too, sorry about that I
What options are you running Uglify with, and what version are you
running?  I’m using "uglify boot.js -m -c”, I just timed it on mine
and it’s mangled the qxt app in 11 seconds (I’m using uglify 2.6.1)
 Still not great but better than 30 secs :)  I’ve nearly got Uglify
done, but stopped for dinner and then saw your email so switched
back.
I’ve done the #! entries too
I’ve pushed a new release, give it a go :)
John
Date: Wednesday, 17 February 2016 at 19:33
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster
compilation, and 100% Javascript API to building applications
Just a quick follow-up on Babel - I've rolled back to yesterday's
version, edited lib/qxcompiler/ClassFile.js:218, replaced that
plugins array with presets: [ "es2015" ]. Suddenly, it worked :)
I'm still wondering why it doesn't try to look for any .babelrc file.
It's clearly stated in the docs that the "babelrc" option is true by
default. And it definitely would be good to have control over Babel
via (optional) config file.
Another issue to consider is that not every ES6 feature is
implemented via code transformation. Some (ex., Promises) require
babel-polyfill. (Mention in the docs? Integrate polyfill into build?)
Dimitri
Post by Dietrich Streifert
Hi John,
That pace you're keeping, it's incredible :) Glad to hear
translations finally landed!
Unfortunately, I couldn't make them work :(
[BABEL] Note: The code generator has deoptimised the styling of
"/home/mitya/qxcompiler/qooxdoo/framework/source/class/qx/ui/core/W
idget.js" as it exceeds the max of "100KB".
2016-02-17 21:30:13.229 [info ] makers          Writing target
Source Target: ../testdata/qxt/source-output/
/home/mitya/qxcompiler/lib/qxcompiler/targets/Target.js:239
              pkgdata.locales["C"] = db.cldr["en"];
                                            ^
TypeError: Cannot read property 'en' of undefined
    at
/home/mitya/qxcompiler/lib/qxcompiler/targets/Target.js:239:45
    at
/home/mitya/qxcompiler/node_modules/async/lib/async.js:718:13
    at async.forEachOf.async.eachOf
(/home/mitya/qxcompiler/node_modules/async/lib/async.js:233:13)
    at _parallel
(/home/mitya/qxcompiler/node_modules/async/lib/async.js:717:9)
    at Object.async.parallel
(/home/mitya/qxcompiler/node_modules/async/lib/async.js:731:9)
    at
/home/mitya/qxcompiler/lib/qxcompiler/targets/Target.js:236:15
    at /home/mitya/qxcompiler/lib/util.js:321:9
    at /home/mitya/qxcompiler/node_modules/async/lib/async.js:52:16
    at
/home/mitya/qxcompiler/node_modules/async/lib/async.js:269:32
    at /home/mitya/qxcompiler/node_modules/async/lib/async.js:44:16
(removed qxcompiler dir & did clean checkout, didn't help either)
Re: Babel - sadly, didn't make it work either :( that arrow
function in qxt/Application.js goes unchanged first to
[source|build]-output/transpiled, then to [source|build]-
output/script. The same with string templates, etc. Note the above
[BABEL] message - that means, Babel is invoked, but doesn't apply
any actual transformations. I tried "strace -e trace=open node
compile-app-demo.js", and didn't see it open .babelrc or any plugin
and/or preset modules. Could you please make sure it's indeed
enabled? If it works for you, could you please assist me in
tracing/debugging (since I'm a newbie in a Node world)?
As for script compression, I played a bit with the standalone
uglify tool. Works pretty fine, but it takes ~30 seconds to process
a skeleton app. Do you think it can be sped up if integrated into
QxCompiler?
Minor issue: test/*.js scripts could have a shebang string
(#!/usr/bin/node) and +x mode so that they could be run directly,
without typing "node ..." each time.
Cheers!
Dimitri
-------------------------------------------------------------------
----------- Site24x7 APM Insight: Get Deep Visibility into
Application Performance APM + Mobile APM + RUM: Monitor 3 App
instances at just $35/Month Monitor end-to-end web transactions and
take corrective actions now Troubleshoot faster and improve end-user
experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id
=272487151&iu=/4140_______________________________________________
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
-------------------------------------------------------------------
-----------
Site24x7 APM Insight: Get Deep Visibility into Application
Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
John Spackman
2016-02-17 22:37:39 UTC
Permalink
Hmm - what happens if you change directory to the root of the QxCompiler repo and run ./test/compile-app-demo.js?

From: Dimitri <***@cargosoft.ru>
Reply-To: qooxdoo Development <qooxdoo-***@lists.sourceforge.net>
Date: Wednesday, 17 February 2016 at 22:31
To: <qooxdoo-***@lists.sourceforge.net>
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster compilation, and 100% Javascript API to building applications

No cigar this time :-\

[***@localhost test]$ ./compile-app-demo.js
/home/mitya/qxcompiler/node_modules/babel-core/lib/transformation/file/options/option-manager.js:393
throw new Error("Couldn't find preset " + JSON.stringify(val) + " relative to directory " + JSON.stringify(dirname));
^

Error: Couldn't find preset "es2015" relative to directory "/home/mitya"
at /home/mitya/qxcompiler/node_modules/babel-core/lib/transformation/file/options/option-manager.js:393:17
at Array.map (native)
at OptionManager.resolvePresets (/home/mitya/qxcompiler/node_modules/babel-core/lib/transformation/file/options/option-manager.js:385:20)
at OptionManager.mergePresets (/home/mitya/qxcompiler/node_modules/babel-core/lib/transformation/file/options/option-manager.js:369:10)
at OptionManager.mergeOptions (/home/mitya/qxcompiler/node_modules/babel-core/lib/transformation/file/options/option-manager.js:328:14)
at OptionManager.addConfig (/home/mitya/qxcompiler/node_modules/babel-core/lib/transformation/file/options/option-manager.js:234:10)
at OptionManager.findConfigs (/home/mitya/qxcompiler/node_modules/babel-core/lib/transformation/file/options/option-manager.js:434:16)
at OptionManager.init (/home/mitya/qxcompiler/node_modules/babel-core/lib/transformation/file/options/option-manager.js:482:12)
at File.initOptions (/home/mitya/qxcompiler/node_modules/babel-core/lib/transformation/file/index.js:211:75)
at new File (/home/mitya/qxcompiler/node_modules/babel-core/lib/transformation/file/index.js:129:22)

Dunno why it looks for presets in my home dir. In my previous experiment, everything went smooth (except that it didn't in fact collect any qooxdoo deps, since I removed all plugins - but transpiling itself went fine).

As for uglify - ah, that was stupid. I used original uglify, not uglify-js. Just tried the latter (2.6.1) - indeed, it takes ~10 secs for qxt. Not bad.

В Ср, 17/02/2016 в 20:56 +0000, John Spackman пОшет:
Got it – it was a bug in my plugin I introduced a while ago, but Chrome obviously supports arrow functions natively so I didn’t pick up that the translation wasn’t happening any more. The .babelrc is not needed for this because we’re using it embedded, I think the .babelrc is only for the CLI

That’s very a good point about the polypill BTW because it’s not being included and it should be, I’ll add that in tomorrow.

The problem with translations is fixed now too, sorry about that I should have deleted the output directory before retesting :oops:

What options are you running Uglify with, and what version are you running? I’m using "uglify boot.js -m -c”, I just timed it on mine and it’s mangled the qxt app in 11 seconds (I’m using uglify 2.6.1) Still not great but better than 30 secs :) I’ve nearly got Uglify done, but stopped for dinner and then saw your email so switched back.

I’ve done the #! entries too

I’ve pushed a new release, give it a go :)

John

From: Dimitri <***@cargosoft.ru>
Reply-To: qooxdoo Development <qooxdoo-***@lists.sourceforge.net>
Date: Wednesday, 17 February 2016 at 19:33
To: <qooxdoo-***@lists.sourceforge.net>
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster compilation, and 100% Javascript API to building applications

Just a quick follow-up on Babel - I've rolled back to yesterday's version, edited lib/qxcompiler/ClassFile.js:218, replaced that plugins array with presets: [ "es2015" ]. Suddenly, it worked :)
I'm still wondering why it doesn't try to look for any .babelrc file. It's clearly stated in the docs that the "babelrc" option is true by default. And it definitely would be good to have control over Babel via (optional) config file.

Another issue to consider is that not every ES6 feature is implemented via code transformation. Some (ex., Promises) require babel-polyfill. (Mention in the docs? Integrate polyfill into build?)

Dimitri

Hi John,

That pace you're keeping, it's incredible :) Glad to hear translations finally landed!

Unfortunately, I couldn't make them work :(

[***@localhost test]$ node compile-app-demo.js
[BABEL] Note: The code generator has deoptimised the styling of "/home/mitya/qxcompiler/qooxdoo/framework/source/class/qx/ui/core/Widget.js" as it exceeds the max of "100KB".
2016-02-17 21:30:13.229 [info ] makers Writing target Source Target: ../testdata/qxt/source-output/
/home/mitya/qxcompiler/lib/qxcompiler/targets/Target.js:239
pkgdata.locales["C"] = db.cldr["en"];
^

TypeError: Cannot read property 'en' of undefined
at /home/mitya/qxcompiler/lib/qxcompiler/targets/Target.js:239:45
at /home/mitya/qxcompiler/node_modules/async/lib/async.js:718:13
at async.forEachOf.async.eachOf (/home/mitya/qxcompiler/node_modules/async/lib/async.js:233:13)
at _parallel (/home/mitya/qxcompiler/node_modules/async/lib/async.js:717:9)
at Object.async.parallel (/home/mitya/qxcompiler/node_modules/async/lib/async.js:731:9)
at /home/mitya/qxcompiler/lib/qxcompiler/targets/Target.js:236:15
at /home/mitya/qxcompiler/lib/util.js:321:9
at /home/mitya/qxcompiler/node_modules/async/lib/async.js:52:16
at /home/mitya/qxcompiler/node_modules/async/lib/async.js:269:32
at /home/mitya/qxcompiler/node_modules/async/lib/async.js:44:16

(removed qxcompiler dir & did clean checkout, didn't help either)

Re: Babel - sadly, didn't make it work either :( that arrow function in qxt/Application.js goes unchanged first to [source|build]-output/transpiled, then to [source|build]-output/script. The same with string templates, etc. Note the above [BABEL] message - that means, Babel is invoked, but doesn't apply any actual transformations. I tried "strace -e trace=open node compile-app-demo.js", and didn't see it open .babelrc or any plugin and/or preset modules. Could you please make sure it's indeed enabled? If it works for you, could you please assist me in tracing/debugging (since I'm a newbie in a Node world)?

As for script compression, I played a bit with the standalone uglify tool. Works pretty fine, but it takes ~30 seconds to process a skeleton app. Do you think it can be sped up if integrated into QxCompiler?

Minor issue: test/*.js scripts could have a shebang string (#!/usr/bin/node) and +x mode so that they could be run directly, without typing "node ..." each time.

Cheers!
Dimitri
------------------------------------------------------------------------------ Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140_______________________________________________ qooxdoo-devel mailing list qooxdoo-***@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
Dimitri
2016-02-17 23:02:35 UTC
Permalink
[***@localhost qxcompiler]$ ./test/compile-app-demo.js
Error: Error: ENOENT: no such file or directory, open
'../testdata/qxt/Manifest.json'
Post by John Spackman
Hmm - what happens if you change directory to the root of the
QxCompiler repo and run ./test/compile-app-demo.js?
Date: Wednesday, 17 February 2016 at 22:31
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster
compilation, and 100% Javascript API to building applications
No cigar this time :-\
/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/options/option-manager.js:393
          throw new Error("Couldn't find preset " +
JSON.stringify(val) + " relative to directory " +
JSON.stringify(dirname));
          ^
Error: Couldn't find preset "es2015" relative to directory
"/home/mitya"
    at /home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/options/option-manager.js:393:17
    at Array.map (native)
    at OptionManager.resolvePresets
(/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/options/option-manager.js:385:20)
    at OptionManager.mergePresets
(/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/options/option-manager.js:369:10)
    at OptionManager.mergeOptions
(/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/options/option-manager.js:328:14)
    at OptionManager.addConfig
(/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/options/option-manager.js:234:10)
    at OptionManager.findConfigs
(/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/options/option-manager.js:434:16)
    at OptionManager.init (/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/options/option-manager.js:482:12)
    at File.initOptions (/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/index.js:211:75)
    at new File (/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/index.js:129:22)
Dunno why it looks for presets in my home dir. In my previous
experiment, everything went smooth (except that it didn't in fact
collect any qooxdoo deps, since I removed all plugins - but
transpiling itself went fine).
As for uglify - ah, that was stupid. I used original uglify, not
uglify-js. Just tried the latter (2.6.1) - indeed, it takes ~10 secs
for qxt. Not bad.
Post by John Spackman
Got it – it was a bug in my plugin I introduced a while ago, but
Chrome obviously supports arrow functions natively so I didn’t pick
up that the translation wasn’t happening any more.  The .babelrc is
not needed for this because we’re using it embedded, I think the
.babelrc is only for the CLI
That’s very a good point about the polypill BTW because it’s not
being included and it should be, I’ll add that in tomorrow.
The problem with translations is fixed now too, sorry about that I
What options are you running Uglify with, and what version are you
running?  I’m using "uglify boot.js -m -c”, I just timed it on mine
and it’s mangled the qxt app in 11 seconds (I’m using uglify 2.6.1)
 Still not great but better than 30 secs :)  I’ve nearly got Uglify
done, but stopped for dinner and then saw your email so switched
back.
I’ve done the #! entries too
I’ve pushed a new release, give it a go :)
John
Date: Wednesday, 17 February 2016 at 19:33
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster
compilation, and 100% Javascript API to building applications
Just a quick follow-up on Babel - I've rolled back to yesterday's
version, edited lib/qxcompiler/ClassFile.js:218, replaced that
plugins array with presets: [ "es2015" ]. Suddenly, it worked :)
I'm still wondering why it doesn't try to look for any .babelrc
file. It's clearly stated in the docs that the "babelrc" option is
true by default. And it definitely would be good to have control
over Babel via (optional) config file.
Another issue to consider is that not every ES6 feature is
implemented via code transformation. Some (ex., Promises) require
babel-polyfill. (Mention in the docs? Integrate polyfill into build?)
Dimitri
Post by Dietrich Streifert
Hi John,
That pace you're keeping, it's incredible :) Glad to hear
translations finally landed!
Unfortunately, I couldn't make them work :(
[BABEL] Note: The code generator has deoptimised the styling of
"/home/mitya/qxcompiler/qooxdoo/framework/source/class/qx/ui/core
/Widget.js" as it exceeds the max of "100KB".
2016-02-17 21:30:13.229 [info ] makers          Writing target
Source Target: ../testdata/qxt/source-output/
/home/mitya/qxcompiler/lib/qxcompiler/targets/Target.js:239
              pkgdata.locales["C"] = db.cldr["en"];
                                            ^
TypeError: Cannot read property 'en' of undefined
    at
/home/mitya/qxcompiler/lib/qxcompiler/targets/Target.js:239:45
    at
/home/mitya/qxcompiler/node_modules/async/lib/async.js:718:13
    at async.forEachOf.async.eachOf
(/home/mitya/qxcompiler/node_modules/async/lib/async.js:233:13)
    at _parallel
(/home/mitya/qxcompiler/node_modules/async/lib/async.js:717:9)
    at Object.async.parallel
(/home/mitya/qxcompiler/node_modules/async/lib/async.js:731:9)
    at
/home/mitya/qxcompiler/lib/qxcompiler/targets/Target.js:236:15
    at /home/mitya/qxcompiler/lib/util.js:321:9
    at
/home/mitya/qxcompiler/node_modules/async/lib/async.js:52:16
    at
/home/mitya/qxcompiler/node_modules/async/lib/async.js:269:32
    at
/home/mitya/qxcompiler/node_modules/async/lib/async.js:44:16
(removed qxcompiler dir & did clean checkout, didn't help either)
Re: Babel - sadly, didn't make it work either :( that arrow
function in qxt/Application.js goes unchanged first to
[source|build]-output/transpiled, then to [source|build]-
output/script. The same with string templates, etc. Note the
above [BABEL] message - that means, Babel is invoked, but doesn't
apply any actual transformations. I tried "strace -e trace=open
node compile-app-demo.js", and didn't see it open .babelrc or any
plugin and/or preset modules. Could you please make sure it's
indeed enabled? If it works for you, could you please assist me
in tracing/debugging (since I'm a newbie in a Node world)?
As for script compression, I played a bit with the standalone
uglify tool. Works pretty fine, but it takes ~30 seconds to
process a skeleton app. Do you think it can be sped up if
integrated into QxCompiler?
Minor issue: test/*.js scripts could have a shebang string
(#!/usr/bin/node) and +x mode so that they could be run directly,
without typing "node ..." each time.
Cheers!
Dimitri
-----------------------------------------------------------------
------------- Site24x7 APM Insight: Get Deep Visibility into
Application Performance APM + Mobile APM + RUM: Monitor 3 App
instances at just $35/Month Monitor end-to-end web transactions and
take corrective actions now Troubleshoot faster and improve end-
user experience. Signup Now! http://pubads.g.doubleclick.net/gampad
/clk?id=272487151&iu=/4140_________________________________________
______ qooxdoo-devel mailing list qooxdoo-
info/qooxdoo-devel
-----------------------------------------------------------------
-------------
Site24x7 APM Insight: Get Deep Visibility into Application
Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
sts/listinfo/qooxdoo-devel
-------------------------------------------------------------------
----------- Site24x7 APM Insight: Get Deep Visibility into
Application Performance APM + Mobile APM + RUM: Monitor 3 App
instances at just $35/Month Monitor end-to-end web transactions and
take corrective actions now Troubleshoot faster and improve end-user
experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id
=272487151&iu=/4140_______________________________________________
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
-------------------------------------------------------------------
-----------
Site24x7 APM Insight: Get Deep Visibility into Application
Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
John Spackman
2016-02-18 05:40:34 UTC
Permalink
Hi Dimitri

Sorry I realise now that was a rash suggestion, obviously it had to be in the test directory to work.

I think there’s something odd with your installation because I’ve just switched to a Centos 6.5 VM that has never had node installed, installed Node 4, ImageMagick, cloned the QxCompiler repo, and run npm install. I discovered a problem with package.json where babel-types wasn’t listed, but once I fixed that with "npm install babel-types —save” I was able to “cd test ; ./compile-app-demo.js” and get a working app, with resources and Bable transpiling.

I tried Node 5 too on the same machine (deleting my ~/.npm, node_modules, and testdata/qxt/source-output directories first) and that also works for me; also the ./compile-app-demo-build.js

If I list installed plugins I get:
$ npm list | grep 2015
├─┬ babel-preset-***@6.5.0
│ ├─┬ babel-plugin-check-es2015-***@6.5.0
│ ├─┬ babel-plugin-transform-es2015-arrow-***@6.5.2
│ ├─┬ babel-plugin-transform-es2015-block-scoped-***@6.5.0
│ ├─┬ babel-plugin-transform-es2015-block-***@6.5.0
│ ├─┬ babel-plugin-transform-es2015-***@6.5.2
│ ├─┬ babel-plugin-transform-es2015-computed-***@6.5.2
│ ├─┬ babel-plugin-transform-es2015-***@6.5.0
│ ├─┬ babel-plugin-transform-es2015-for-***@6.5.2
│ ├─┬ babel-plugin-transform-es2015-function-***@6.5.0
│ ├─┬ babel-plugin-transform-es2015-***@6.5.0
│ ├─┬ babel-plugin-transform-es2015-modules-***@6.5.2
│ ├─┬ babel-plugin-transform-es2015-object-***@6.5.0
│ ├─┬ babel-plugin-transform-es2015-***@6.5.0
│ ├─┬ babel-plugin-transform-es2015-shorthand-***@6.5.0
│ ├─┬ babel-plugin-transform-es2015-***@6.5.2
│ ├─┬ babel-plugin-transform-es2015-sticky-***@6.5.0
│ ├─┬ babel-plugin-transform-es2015-template-***@6.5.2
│ ├─┬ babel-plugin-transform-es2015-typeof-***@6.5.0
│ ├─┬ babel-plugin-transform-es2015-unicode-***@6.5.0

The array of plugins that was in QxCompiler previously was the equivalent of the es2015 preset, albeit the list was slightly out of date so updating the code to use a preset instead of plugins should only improve support, but you had it working with the preset yesterday so I don’t understand why it is not working now.

Is it possible that you have Babel installed globally and it’s conflicting somehow? I’ve updated the package.json at GitHub to explicitly reference babel-types, and rechecking out etc on my Centos 6 VM now runs without any complaints so it might be worth updating and trying again.

Regards
John

From: Dimitri <***@cargosoft.ru>
Reply-To: qooxdoo Development <qooxdoo-***@lists.sourceforge.net>
Date: Wednesday, 17 February 2016 at 23:02
To: <qooxdoo-***@lists.sourceforge.net>
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster compilation, and 100% Javascript API to building applications

[***@localhost qxcompiler]$ ./test/compile-app-demo.js
Error: Error: ENOENT: no such file or directory, open '../testdata/qxt/Manifest.json'


Hmm - what happens if you change directory to the root of the QxCompiler repo and run ./test/compile-app-demo.js?

From: Dimitri <***@cargosoft.ru>
Reply-To: qooxdoo Development <qooxdoo-***@lists.sourceforge.net>
Date: Wednesday, 17 February 2016 at 22:31
To: <qooxdoo-***@lists.sourceforge.net>
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster compilation, and 100% Javascript API to building applications

No cigar this time :-\

[***@localhost test]$ ./compile-app-demo.js
/home/mitya/qxcompiler/node_modules/babel-core/lib/transformation/file/options/option-manager.js:393
throw new Error("Couldn't find preset " + JSON.stringify(val) + " relative to directory " + JSON.stringify(dirname));
^

Error: Couldn't find preset "es2015" relative to directory "/home/mitya"
at /home/mitya/qxcompiler/node_modules/babel-core/lib/transformation/file/options/option-manager.js:393:17
at Array.map (native)
at OptionManager.resolvePresets (/home/mitya/qxcompiler/node_modules/babel-core/lib/transformation/file/options/option-manager.js:385:20)
at OptionManager.mergePresets (/home/mitya/qxcompiler/node_modules/babel-core/lib/transformation/file/options/option-manager.js:369:10)
at OptionManager.mergeOptions (/home/mitya/qxcompiler/node_modules/babel-core/lib/transformation/file/options/option-manager.js:328:14)
at OptionManager.addConfig (/home/mitya/qxcompiler/node_modules/babel-core/lib/transformation/file/options/option-manager.js:234:10)
at OptionManager.findConfigs (/home/mitya/qxcompiler/node_modules/babel-core/lib/transformation/file/options/option-manager.js:434:16)
at OptionManager.init (/home/mitya/qxcompiler/node_modules/babel-core/lib/transformation/file/options/option-manager.js:482:12)
at File.initOptions (/home/mitya/qxcompiler/node_modules/babel-core/lib/transformation/file/index.js:211:75)
at new File (/home/mitya/qxcompiler/node_modules/babel-core/lib/transformation/file/index.js:129:22)

Dunno why it looks for presets in my home dir. In my previous experiment, everything went smooth (except that it didn't in fact collect any qooxdoo deps, since I removed all plugins - but transpiling itself went fine).

As for uglify - ah, that was stupid. I used original uglify, not uglify-js. Just tried the latter (2.6.1) - indeed, it takes ~10 secs for qxt. Not bad.

В Ср, 17/02/2016 в 20:56 +0000, John Spackman пОшет:
Got it – it was a bug in my plugin I introduced a while ago, but Chrome obviously supports arrow functions natively so I didn’t pick up that the translation wasn’t happening any more. The .babelrc is not needed for this because we’re using it embedded, I think the .babelrc is only for the CLI

That’s very a good point about the polypill BTW because it’s not being included and it should be, I’ll add that in tomorrow.

The problem with translations is fixed now too, sorry about that I should have deleted the output directory before retesting :oops:

What options are you running Uglify with, and what version are you running? I’m using "uglify boot.js -m -c”, I just timed it on mine and it’s mangled the qxt app in 11 seconds (I’m using uglify 2.6.1) Still not great but better than 30 secs :) I’ve nearly got Uglify done, but stopped for dinner and then saw your email so switched back.

I’ve done the #! entries too

I’ve pushed a new release, give it a go :)

John

From: Dimitri <***@cargosoft.ru>
Reply-To: qooxdoo Development <qooxdoo-***@lists.sourceforge.net>
Date: Wednesday, 17 February 2016 at 19:33
To: <qooxdoo-***@lists.sourceforge.net>
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster compilation, and 100% Javascript API to building applications

Just a quick follow-up on Babel - I've rolled back to yesterday's version, edited lib/qxcompiler/ClassFile.js:218, replaced that plugins array with presets: [ "es2015" ]. Suddenly, it worked :)
I'm still wondering why it doesn't try to look for any .babelrc file. It's clearly stated in the docs that the "babelrc" option is true by default. And it definitely would be good to have control over Babel via (optional) config file.

Another issue to consider is that not every ES6 feature is implemented via code transformation. Some (ex., Promises) require babel-polyfill. (Mention in the docs? Integrate polyfill into build?)

Dimitri

Hi John,

That pace you're keeping, it's incredible :) Glad to hear translations finally landed!

Unfortunately, I couldn't make them work :(

[***@localhost test]$ node compile-app-demo.js
[BABEL] Note: The code generator has deoptimised the styling of "/home/mitya/qxcompiler/qooxdoo/framework/source/class/qx/ui/core/Widget.js" as it exceeds the max of "100KB".
2016-02-17 21:30:13.229 [info ] makers Writing target Source Target: ../testdata/qxt/source-output/
/home/mitya/qxcompiler/lib/qxcompiler/targets/Target.js:239
pkgdata.locales["C"] = db.cldr["en"];
^

TypeError: Cannot read property 'en' of undefined
at /home/mitya/qxcompiler/lib/qxcompiler/targets/Target.js:239:45
at /home/mitya/qxcompiler/node_modules/async/lib/async.js:718:13
at async.forEachOf.async.eachOf (/home/mitya/qxcompiler/node_modules/async/lib/async.js:233:13)
at _parallel (/home/mitya/qxcompiler/node_modules/async/lib/async.js:717:9)
at Object.async.parallel (/home/mitya/qxcompiler/node_modules/async/lib/async.js:731:9)
at /home/mitya/qxcompiler/lib/qxcompiler/targets/Target.js:236:15
at /home/mitya/qxcompiler/lib/util.js:321:9
at /home/mitya/qxcompiler/node_modules/async/lib/async.js:52:16
at /home/mitya/qxcompiler/node_modules/async/lib/async.js:269:32
at /home/mitya/qxcompiler/node_modules/async/lib/async.js:44:16

(removed qxcompiler dir & did clean checkout, didn't help either)

Re: Babel - sadly, didn't make it work either :( that arrow function in qxt/Application.js goes unchanged first to [source|build]-output/transpiled, then to [source|build]-output/script. The same with string templates, etc. Note the above [BABEL] message - that means, Babel is invoked, but doesn't apply any actual transformations. I tried "strace -e trace=open node compile-app-demo.js", and didn't see it open .babelrc or any plugin and/or preset modules. Could you please make sure it's indeed enabled? If it works for you, could you please assist me in tracing/debugging (since I'm a newbie in a Node world)?

As for script compression, I played a bit with the standalone uglify tool. Works pretty fine, but it takes ~30 seconds to process a skeleton app. Do you think it can be sped up if integrated into QxCompiler?

Minor issue: test/*.js scripts could have a shebang string (#!/usr/bin/node) and +x mode so that they could be run directly, without typing "node ..." each time.

Cheers!
Dimitri
------------------------------------------------------------------------------ Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140_______________________________________________ qooxdoo-devel mailing list qooxdoo-***@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
qooxdoo-***@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
------------------------------------------------------------------------------ Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140_______________________________________________ qooxdoo-devel mailing list qooxdoo-***@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
Dimitri
2016-02-18 08:16:54 UTC
Permalink
John,
This turns to be one of that "bug reports that you like to hear" ;)
I've identified the culprit, it was .babelrc in my home dir. We were
totally wrong in that Babel ignores .babelrc in embedded mode. In fact,
it looks for it *everywhere*:
[***@localhost test]$ strace node ./compile-app-demo.js 2>&1 | grep
babelrc
access("/home/mitya/qxcompiler/testdata/qxt/source/class/qxt/.babelrc",
F_OK) = -1 ENOENT (No such file or directory)
access("/home/mitya/qxcompiler/testdata/qxt/source/class/.babelrc",
F_OK) = -1 ENOENT (No such file or directory)
access("/home/mitya/qxcompiler/testdata/qxt/source/.babelrc", F_OK) =
-1 ENOENT (No such file or directory)
access("/home/mitya/qxcompiler/testdata/qxt/.babelrc", F_OK) = -1
ENOENT (No such file or directory)
access("/home/mitya/qxcompiler/testdata/.babelrc", F_OK) = -1 ENOENT
(No such file or directory)
access("/home/mitya/qxcompiler/.babelrc", F_OK) = -1 ENOENT (No such
file or directory)
access("/home/mitya/.babelrc", F_OK) = 0
open("/home/mitya/.babelrc", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 11After the file has been encountered, seems like it tries to resolve presets/plugins relative to that file location.
With that file removed, everything finally worked with qxt! But then I tried to build my own project, and got similar error:
/home/mitya/qxcompiler/node_modules/babel-core/lib/transformation/file/options/option-manager.js:393
throw new Error("Couldn't find preset " + JSON.stringify(val) + " relative to directory " + JSON.stringify(dirname));
^

Error: Couldn't find preset "es2015" relative to directory "/home/mitya/Projects/foo/source/class/foo"
at /home/mitya/qxcompiler/node_modules/babel-core/lib/transformation/file/options/option-manager.js:393:17
at Array.map (native)
at OptionManager.resolvePresets (/home/mitya/qxcompiler/node_modules/babel-core/lib/transformation/file/options/option-manager.js:385:20)
at OptionManager.mergePresets (/home/mitya/qxcompiler/node_modules/babel-core/lib/transformation/file/options/option-manager.js:369:10)
at OptionManager.mergeOptions (/home/mitya/qxcompiler/node_modules/babel-core/lib/transformation/file/options/option-manager.js:328:14)
at OptionManager.init (/home/mitya/qxcompiler/node_modules/babel-core/lib/transformation/file/options/option-manager.js:486:10)
at File.initOptions (/home/mitya/qxcompiler/node_modules/babel-core/lib/transformation/file/index.js:211:75)
at new File (/home/mitya/qxcompiler/node_modules/babel-core/lib/transformation/file/index.js:129:22)
at Pipeline.transform (/home/mitya/qxcompiler/node_modules/babel-core/lib/transformation/pipeline.js:48:16)
at /home/mitya/qxcompiler/lib/qxcompiler/ClassFile.js:220:32

This time no .babelrc files anywhere in the project structure. Remember I earlier did "npm link async" in the project dir so that build script would run. To get rid of this, I removed foo/node_modules and symlinked qxcompiler/node_modules to the project dir. Got yet another error, this time with UploadMgr:
Error: Couldn't find preset "es2015" relative to directory "/home/mitya/qooxdoo-contrib/UploadMgr/source/class/com/zenesis/qx/upload"
at /home/mitya/qxcompiler/node_modules/babel-core/lib/transformation/file/options/option-manager.js:393:17
at Array.map (native)
at OptionManager.resolvePresets (/home/mitya/qxcompiler/node_modules/babel-core/lib/transformation/file/options/option-manager.js:385:20)
at OptionManager.mergePresets (/home/mitya/qxcompiler/node_modules/babel-core/lib/transformation/file/options/option-manager.js:369:10)
at OptionManager.mergeOptions (/home/mitya/qxcompiler/node_modules/babel-core/lib/transformation/file/options/option-manager.js:328:14)
at OptionManager.init (/home/mitya/qxcompiler/node_modules/babel-core/lib/transformation/file/options/option-manager.js:486:10)
at File.initOptions (/home/mitya/qxcompiler/node_modules/babel-core/lib/transformation/file/index.js:211:75)
at new File (/home/mitya/qxcompiler/node_modules/babel-core/lib/transformation/file/index.js:129:22)
at Pipeline.transform (/home/mitya/qxcompiler/node_modules/babel-core/lib/transformation/pipeline.js:48:16)
at /home/mitya/qxcompiler/lib/qxcompiler/ClassFile.js:220:32

With node_modules symlinked to qooxdoo-contrib/UploadMgr, finally everything worked. This is easily reproducible if you move qxt outside of QxCompiler dir, place build script inside qxt and adjust paths accordingly (I used absolute paths).
Transpiling and compression work just fine. Yet I've encountered a small bug with translations. If a message is translated with this.tr*() set of functions, everything works fine. If qx.locale.Manager.tr() is used, such a message simply won't make it into the build.
I think we should unconditionally transfer all the *.po strings into the build, since translation in the app could be done based on dynamic, computed values.
I think that as soon as the remaining issues are fixed, QxCompiler can be considered fairly stable. I'm already integrating it into our build process.
Cheers!
Dimitri
Post by John Spackman
Hi Dimitri
Sorry I realise now that was a rash suggestion, obviously it had to
be in the test directory to work.
I think there’s something odd with your installation because I’ve
just switched to a Centos 6.5 VM that has never had node installed,
installed Node 4, ImageMagick, cloned the QxCompiler repo, and run
npm install.  I discovered a problem with package.json where babel-
types wasn’t listed, but once I fixed that with "npm install babel-
types —save” I was able to “cd test ; ./compile-app-demo.js” and get
a working app, with resources and Bable transpiling.
I tried Node 5 too on the same machine (deleting my ~/.npm,
node_modules, and testdata/qxt/source-output directories first) and
that also works for me; also the ./compile-app-demo-build.js
$ npm list | grep 2015
The array of plugins that was in QxCompiler previously was the
equivalent of the es2015 preset, albeit the list was slightly out of
date so updating the code to use a preset instead of plugins should
only improve support, but you had it working with the preset
yesterday so I don’t understand why it is not working now.
Is it possible that you have Babel installed globally and it’s
conflicting somehow?  I’ve updated the package.json at GitHub to
explicitly reference babel-types, and rechecking out etc on my Centos
6 VM now runs without any complaints so it might be worth updating
and trying again.
Regards
John
Date: Wednesday, 17 February 2016 at 23:02
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster
compilation, and 100% Javascript API to building applications
Error: Error: ENOENT: no such file or directory, open
'../testdata/qxt/Manifest.json'
Post by John Spackman
Hmm - what happens if you change directory to the root of the
QxCompiler repo and run ./test/compile-app-demo.js?
Date: Wednesday, 17 February 2016 at 22:31
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster
compilation, and 100% Javascript API to building applications
No cigar this time :-\
/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/options/option-manager.js:393
          throw new Error("Couldn't find preset " +
JSON.stringify(val) + " relative to directory " +
JSON.stringify(dirname));
          ^
Error: Couldn't find preset "es2015" relative to directory
"/home/mitya"
    at /home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/options/option-manager.js:393:17
    at Array.map (native)
    at OptionManager.resolvePresets
(/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/options/option-manager.js:385:20)
    at OptionManager.mergePresets
(/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/options/option-manager.js:369:10)
    at OptionManager.mergeOptions
(/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/options/option-manager.js:328:14)
    at OptionManager.addConfig
(/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/options/option-manager.js:234:10)
    at OptionManager.findConfigs
(/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/options/option-manager.js:434:16)
    at OptionManager.init
(/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/options/option-manager.js:482:12)
    at File.initOptions (/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/index.js:211:75)
    at new File (/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/index.js:129:22)
Dunno why it looks for presets in my home dir. In my previous
experiment, everything went smooth (except that it didn't in fact
collect any qooxdoo deps, since I removed all plugins - but
transpiling itself went fine).
As for uglify - ah, that was stupid. I used original uglify, not
uglify-js. Just tried the latter (2.6.1) - indeed, it takes ~10
secs for qxt. Not bad.
Post by John Spackman
Got it – it was a bug in my plugin I introduced a while ago, but
Chrome obviously supports arrow functions natively so I didn’t
pick up that the translation wasn’t happening any more.  The
.babelrc is not needed for this because we’re using it embedded,
I think the .babelrc is only for the CLI
That’s very a good point about the polypill BTW because it’s not
being included and it should be, I’ll add that in tomorrow.
The problem with translations is fixed now too, sorry about that
I should have deleted the output directory before retesting
What options are you running Uglify with, and what version are
you running?  I’m using "uglify boot.js -m -c”, I just timed it
on mine and it’s mangled the qxt app in 11 seconds (I’m using
uglify 2.6.1)  Still not great but better than 30 secs :)  I’ve
nearly got Uglify done, but stopped for dinner and then saw your
email so switched back.
I’ve done the #! entries too
I’ve pushed a new release, give it a go :)
John
Reply-To: qooxdoo Development
t>
Date: Wednesday, 17 February 2016 at 19:33
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster
compilation, and 100% Javascript API to building applications
Just a quick follow-up on Babel - I've rolled back to yesterday's
version, edited lib/qxcompiler/ClassFile.js:218, replaced that
plugins array with presets: [ "es2015" ]. Suddenly, it worked :)
I'm still wondering why it doesn't try to look for any .babelrc
file. It's clearly stated in the docs that the "babelrc" option
is true by default. And it definitely would be good to have
control over Babel via (optional) config file.
Another issue to consider is that not every ES6 feature is
implemented via code transformation. Some (ex., Promises) require
babel-polyfill. (Mention in the docs? Integrate polyfill into build?)
Dimitri
Post by Dietrich Streifert
Hi John,
That pace you're keeping, it's incredible :) Glad to hear
translations finally landed!
Unfortunately, I couldn't make them work :(
[BABEL] Note: The code generator has deoptimised the styling of
"/home/mitya/qxcompiler/qooxdoo/framework/source/class/qx/ui/co
re/Widget.js" as it exceeds the max of "100KB".
2016-02-17 21:30:13.229 [info ] makers          Writing target
Source Target: ../testdata/qxt/source-output/
/home/mitya/qxcompiler/lib/qxcompiler/targets/Target.js:239
              pkgdata.locales["C"] = db.cldr["en"];
                                            ^
TypeError: Cannot read property 'en' of undefined
    at
/home/mitya/qxcompiler/lib/qxcompiler/targets/Target.js:239:45
    at
/home/mitya/qxcompiler/node_modules/async/lib/async.js:718:13
    at async.forEachOf.async.eachOf
(/home/mitya/qxcompiler/node_modules/async/lib/async.js:233:13)
    at _parallel
(/home/mitya/qxcompiler/node_modules/async/lib/async.js:717:9)
    at Object.async.parallel
(/home/mitya/qxcompiler/node_modules/async/lib/async.js:731:9)
    at
/home/mitya/qxcompiler/lib/qxcompiler/targets/Target.js:236:15
    at /home/mitya/qxcompiler/lib/util.js:321:9
    at
/home/mitya/qxcompiler/node_modules/async/lib/async.js:52:16
    at
/home/mitya/qxcompiler/node_modules/async/lib/async.js:269:32
    at
/home/mitya/qxcompiler/node_modules/async/lib/async.js:44:16
(removed qxcompiler dir & did clean checkout, didn't help either)
Re: Babel - sadly, didn't make it work either :( that arrow
function in qxt/Application.js goes unchanged first to
[source|build]-output/transpiled, then to [source|build]-
output/script. The same with string templates, etc. Note the
above [BABEL] message - that means, Babel is invoked, but
doesn't apply any actual transformations. I tried "strace -e
trace=open node compile-app-demo.js", and didn't see it open
.babelrc or any plugin and/or preset modules. Could you please
make sure it's indeed enabled? If it works for you, could you
please assist me in tracing/debugging (since I'm a newbie in a
Node world)?
As for script compression, I played a bit with the standalone
uglify tool. Works pretty fine, but it takes ~30 seconds to
process a skeleton app. Do you think it can be sped up if
integrated into QxCompiler?
Minor issue: test/*.js scripts could have a shebang string
(#!/usr/bin/node) and +x mode so that they could be run
directly, without typing "node ..." each time.
Cheers!
Dimitri
---------------------------------------------------------------
--------------- Site24x7 APM Insight: Get Deep Visibility into
Application Performance APM + Mobile APM + RUM: Monitor 3 App
instances at just $35/Month Monitor end-to-end web transactions
and take corrective actions now Troubleshoot faster and improve
end-user experience. Signup Now! http://pubads.g.doubleclick.net/
gampad/clk?id=272487151&iu=/4140_________________________________
______________ qooxdoo-devel mailing list qooxdoo-
stinfo/qooxdoo-devel
---------------------------------------------------------------
---------------
Site24x7 APM Insight: Get Deep Visibility into Application
Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
lists/listinfo/qooxdoo-devel
-----------------------------------------------------------------
------------- Site24x7 APM Insight: Get Deep Visibility into
Application Performance APM + Mobile APM + RUM: Monitor 3 App
instances at just $35/Month Monitor end-to-end web transactions and
take corrective actions now Troubleshoot faster and improve end-
user experience. Signup Now! http://pubads.g.doubleclick.net/gampad
/clk?id=272487151&iu=/4140_________________________________________
______ qooxdoo-devel mailing list qooxdoo-
info/qooxdoo-devel
-----------------------------------------------------------------
-------------
Site24x7 APM Insight: Get Deep Visibility into Application
Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
sts/listinfo/qooxdoo-devel
-------------------------------------------------------------------
----------- Site24x7 APM Insight: Get Deep Visibility into
Application Performance APM + Mobile APM + RUM: Monitor 3 App
instances at just $35/Month Monitor end-to-end web transactions and
take corrective actions now Troubleshoot faster and improve end-user
experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id
=272487151&iu=/4140_______________________________________________
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
-------------------------------------------------------------------
-----------
Site24x7 APM Insight: Get Deep Visibility into Application
Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
John Spackman
2016-02-19 08:04:15 UTC
Permalink
Hi Dimitri

I’ve put it on the todo list to make .babelrc enable-able globally, probably as part of the Maker, so that should be done this weekend too

I did try Nashorn but I had non-trivial issues that I needed to look into - for example, I have a Rhino debugger I’ve written (think node-inspector, but my one is rhino-inspector) and I have to have server side debugging. I couldn’t find much information on how to connect to Nashorn for remote debugging, so couldn’t write a new debugger.

(Rhino-inspector was something I was going to open source at one time but with Nashorn coming on the scene I wondered whether there was any point)

Another problem I had was with Java 8 - trying to use Rhino in Java 8 caused conflicts with Rhino’s implementation of arrays vs Java8, because Rhino attempts to integrate the Java class with Javasacript and there are method conflicts. Again, it looked non-trivial and so I had to put it to one side and am sticking with Java 7 for now.

John
John,
Tried recent version, everything seems to work now. .babelrc's are
ignored; it's no longer needed to symlink node_modules to the app and
contribs directories, "npm link async" in the project dir is
sufficient. However, I have to admit that .babelrc/.babelignore is a
useful mechanism, since (if it worked properly) it would allow to use
different Babel settings, or completely disable it, on a per-directory
basis. Ideally, the compiler should honour .babelrc/.babelignore *and*
pull all the presets and plugins from a single location, be it
qxcompiler/node_modules or /usr/lib/node_modules (as soon as NPM
packaging is ready).
Re: database path - well done! is it safe to use a single shared db
file for a set of projects + particular qooxdoo version (provided that
no parallel compilations occur)?
Wow, we are only one step away from production :) It will be pretty
easy for us to migrate to QxCompiler, since our setup is much simpler
than yours. It's a real-time communication application with pure JavaEE
backend and qooxdoo RIA frontend, so no (yet) JavaScript on server
side.
Slightly offtopic: did you try Nashorn with your server-side? Should be
much faster and feature-rich. Besides, we are planning an opensource
project - JavaScript console for JavaEE with qooxdoo frontend a la
Playground. It would allow to author/run/manage server-side scripts,
with support for CDI injections, scheduled execution, user/system
script library etc. I'd like to ask you (and everyone else here) to let
me know if you're interested.
Dimitri
Post by John Spackman
Hi Dimitri
Well done for finding that, what a nightmare. I think I’ve found how
to disable the .babelrc, but I think the real issue is a bug in Babel
with presets; whatever, the fix is to not use presets and load the
plugins individually which is what I’ve done and I’ve just pushed it
up. Please can you reinstate your .babelrc’s and see if this solves
it?
This release includes the custom database path; translations I’ll get
back onto later on today (I’m a bit behind on my schedule)
That’s great that your using this in production, you’re beating me to
it in mine!
Cheers
John
Date: Thursday, 18 February 2016 at 08:16
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster
compilation, and 100% Javascript API to building applications
John,
This turns to be one of that "bug reports that you like to hear" ;)
I've identified the culprit, it was .babelrc in my home dir. We were
totally wrong in that Babel ignores .babelrc in embedded mode. In
babelrc
access("/home/mitya/qxcompiler/testdata/qxt/source/class/qxt/.babelrc
", F_OK) = -1 ENOENT (No such file or directory)
access("/home/mitya/qxcompiler/testdata/qxt/source/class/.babelrc",
F_OK) = -1 ENOENT (No such file or directory)
access("/home/mitya/qxcompiler/testdata/qxt/source/.babelrc", F_OK) =
-1 ENOENT (No such file or directory)
access("/home/mitya/qxcompiler/testdata/qxt/.babelrc", F_OK) = -1
ENOENT (No such file or directory)
access("/home/mitya/qxcompiler/testdata/.babelrc", F_OK) = -1 ENOENT
(No such file or directory)
access("/home/mitya/qxcompiler/.babelrc", F_OK) = -1 ENOENT (No such
file or directory)
access("/home/mitya/.babelrc", F_OK) = 0
open("/home/mitya/.babelrc", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 11
After the file has been encountered, seems like it tries to resolve
presets/plugins relative to that file location.
With that file removed, everything finally worked with qxt! But then
/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/options/option-manager.js:393
throw new Error("Couldn't find preset " +
JSON.stringify(val) + " relative to directory " +
JSON.stringify(dirname));
^
Error: Couldn't find preset "es2015" relative to directory
"/home/mitya/Projects/foo/source/class/foo"
at /home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/options/option-manager.js:393:17
at Array.map (native)
at OptionManager.resolvePresets
(/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/options/option-manager.js:385:20)
at OptionManager.mergePresets
(/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/options/option-manager.js:369:10)
at OptionManager.mergeOptions
(/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/options/option-manager.js:328:14)
at OptionManager.init (/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/options/option-manager.js:486:10)
at File.initOptions (/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/index.js:211:75)
at new File (/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/index.js:129:22)
at Pipeline.transform (/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/pipeline.js:48:16)
at /home/mitya/qxcompiler/lib/qxcompiler/ClassFile.js:220:32
This time no .babelrc files anywhere in the project structure.
Remember I earlier did "npm link async" in the project dir so that
build script would run. To get rid of this, I removed
foo/node_modules and symlinked qxcompiler/node_modules to the project
Error: Couldn't find preset "es2015" relative to directory
"/home/mitya/qooxdoo-
contrib/UploadMgr/source/class/com/zenesis/qx/upload"
at /home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/options/option-manager.js:393:17
at Array.map (native)
at OptionManager.resolvePresets
(/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/options/option-manager.js:385:20)
at OptionManager.mergePresets
(/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/options/option-manager.js:369:10)
at OptionManager.mergeOptions
(/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/options/option-manager.js:328:14)
at OptionManager.init (/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/options/option-manager.js:486:10)
at File.initOptions (/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/index.js:211:75)
at new File (/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/index.js:129:22)
at Pipeline.transform (/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/pipeline.js:48:16)
at /home/mitya/qxcompiler/lib/qxcompiler/ClassFile.js:220:32
With node_modules symlinked to qooxdoo-contrib/UploadMgr, finally
everything worked. This is easily reproducible if you move qxt
outside of QxCompiler dir, place build script inside qxt and adjust
paths accordingly (I used absolute paths).
Transpiling and compression work just fine. Yet I've encountered a
small bug with translations. If a message is translated with
this.tr*() set of functions, everything works fine. If
qx.locale.Manager.tr() is used, such a message simply won't make it
into the build.
I think we should unconditionally transfer all the *.po strings into
the build, since translation in the app could be done based on
dynamic, computed values.
I think that as soon as the remaining issues are fixed, QxCompiler
can be considered fairly stable. I'm already integrating it into our
build process.
Cheers!
Dimitri
Post by John Spackman
Hi Dimitri
Sorry I realise now that was a rash suggestion, obviously it had to
be in the test directory to work.
I think there’s something odd with your installation because I’ve
just switched to a Centos 6.5 VM that has never had node installed,
installed Node 4, ImageMagick, cloned the QxCompiler repo, and run
npm install. I discovered a problem with package.json where babel-
types wasn’t listed, but once I fixed that with "npm install babel-
types —save” I was able to “cd test ; ./compile-app-demo.js” and
get a working app, with resources and Bable transpiling.
I tried Node 5 too on the same machine (deleting my ~/.npm,
node_modules, and testdata/qxt/source-output directories first) and
that also works for me; also the ./compile-app-demo-build.js
$ npm list | grep 2015
The array of plugins that was in QxCompiler previously was the
equivalent of the es2015 preset, albeit the list was slightly out
of date so updating the code to use a preset instead of plugins
should only improve support, but you had it working with the preset
yesterday so I don’t understand why it is not working now.
Is it possible that you have Babel installed globally and it’s
conflicting somehow? I’ve updated the package.json at GitHub to
explicitly reference babel-types, and rechecking out etc on my
Centos 6 VM now runs without any complaints so it might be worth
updating and trying again.
Regards
John
Date: Wednesday, 17 February 2016 at 23:02
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster
compilation, and 100% Javascript API to building applications
Error: Error: ENOENT: no such file or directory, open
'../testdata/qxt/Manifest.json'
Post by John Spackman
Hmm - what happens if you change directory to the root of the
QxCompiler repo and run ./test/compile-app-demo.js?
t>
Date: Wednesday, 17 February 2016 at 22:31
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster
compilation, and 100% Javascript API to building applications
No cigar this time :-\
/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/options/option-manager.js:393
throw new Error("Couldn't find preset " +
JSON.stringify(val) + " relative to directory " +
JSON.stringify(dirname));
^
Error: Couldn't find preset "es2015" relative to directory "/home/mitya"
at /home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/options/option-manager.js:393:17
at Array.map (native)
at OptionManager.resolvePresets
(/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/options/option-manager.js:385:20)
at OptionManager.mergePresets
(/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/options/option-manager.js:369:10)
at OptionManager.mergeOptions
(/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/options/option-manager.js:328:14)
at OptionManager.addConfig
(/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/options/option-manager.js:234:10)
at OptionManager.findConfigs
(/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/options/option-manager.js:434:16)
at OptionManager.init
(/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/options/option-manager.js:482:12)
at File.initOptions
(/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/index.js:211:75)
at new File (/home/mitya/qxcompiler/node_modules/babel-
core/lib/transformation/file/index.js:129:22)
Dunno why it looks for presets in my home dir. In my previous
experiment, everything went smooth (except that it didn't in fact
collect any qooxdoo deps, since I removed all plugins - but
transpiling itself went fine).
As for uglify - ah, that was stupid. I used original uglify, not
uglify-js. Just tried the latter (2.6.1) - indeed, it takes ~10
secs for qxt. Not bad.
Got it – it was a bug in my plugin I introduced a while ago,
but Chrome obviously supports arrow functions natively so I
didn’t pick up that the translation wasn’t happening any more.
The .babelrc is not needed for this because we’re using it
embedded, I think the .babelrc is only for the CLI
That’s very a good point about the polypill BTW because it’s
not being included and it should be, I’ll add that in tomorrow.
The problem with translations is fixed now too, sorry about
that I should have deleted the output directory before
What options are you running Uglify with, and what version are
you running? I’m using "uglify boot.js -m -c”, I just timed it
on mine and it’s mangled the qxt app in 11 seconds (I’m using
uglify 2.6.1) Still not great but better than 30 secs :) I’ve
nearly got Uglify done, but stopped for dinner and then saw
your email so switched back.
I’ve done the #! entries too
I’ve pushed a new release, give it a go :)
John
net>
Date: Wednesday, 17 February 2016 at 19:33
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster
compilation, and 100% Javascript API to building applications
Just a quick follow-up on Babel - I've rolled back to
yesterday's version, edited lib/qxcompiler/ClassFile.js:218,
replaced that plugins array with presets: [ "es2015" ].
Suddenly, it worked :)
I'm still wondering why it doesn't try to look for any .babelrc
file. It's clearly stated in the docs that the "babelrc" option
is true by default. And it definitely would be good to have
control over Babel via (optional) config file.
Another issue to consider is that not every ES6 feature is
implemented via code transformation. Some (ex., Promises)
require babel-polyfill. (Mention in the docs? Integrate
polyfill into build?)
Dimitri
Post by Dietrich Streifert
Hi John,
That pace you're keeping, it's incredible :) Glad to hear
translations finally landed!
Unfortunately, I couldn't make them work :(
[BABEL] Note: The code generator has deoptimised the styling of
"/home/mitya/qxcompiler/qooxdoo/framework/source/class/qx/ui/
core/Widget.js" as it exceeds the max of "100KB".
2016-02-17 21:30:13.229 [info ] makers Writing
target Source Target: ../testdata/qxt/source-output/
/home/mitya/qxcompiler/lib/qxcompiler/targets/Target.js:239
pkgdata.locales["C"] = db.cldr["en"];
^
TypeError: Cannot read property 'en' of undefined
at
/home/mitya/qxcompiler/lib/qxcompiler/targets/Target.js:239:4
5
at
/home/mitya/qxcompiler/node_modules/async/lib/async.js:718:13
at async.forEachOf.async.eachOf
(/home/mitya/qxcompiler/node_modules/async/lib/async.js:233:1
3)
at _parallel
(/home/mitya/qxcompiler/node_modules/async/lib/async.js:717:9
)
at Object.async.parallel
(/home/mitya/qxcompiler/node_modules/async/lib/async.js:731:9
)
at
/home/mitya/qxcompiler/lib/qxcompiler/targets/Target.js:236:1
5
at /home/mitya/qxcompiler/lib/util.js:321:9
at
/home/mitya/qxcompiler/node_modules/async/lib/async.js:52:16
at
/home/mitya/qxcompiler/node_modules/async/lib/async.js:269:32
at
/home/mitya/qxcompiler/node_modules/async/lib/async.js:44:16
(removed qxcompiler dir & did clean checkout, didn't help either)
Re: Babel - sadly, didn't make it work either :( that arrow
function in qxt/Application.js goes unchanged first to
[source|build]-output/transpiled, then to [source|build]-
output/script. The same with string templates, etc. Note the
above [BABEL] message - that means, Babel is invoked, but
doesn't apply any actual transformations. I tried "strace -e
trace=open node compile-app-demo.js", and didn't see it open
.babelrc or any plugin and/or preset modules. Could you
please make sure it's indeed enabled? If it works for you,
could you please assist me in tracing/debugging (since I'm a
newbie in a Node world)?
As for script compression, I played a bit with the standalone
uglify tool. Works pretty fine, but it takes ~30 seconds to
process a skeleton app. Do you think it can be sped up if
integrated into QxCompiler?
Minor issue: test/*.js scripts could have a shebang string
(#!/usr/bin/node) and +x mode so that they could be run
directly, without typing "node ..." each time.
Cheers!
Dimitri
-------------------------------------------------------------
----------------- Site24x7 APM Insight: Get Deep Visibility
into Application Performance APM + Mobile APM + RUM: Monitor 3
App instances at just $35/Month Monitor end-to-end web
transactions and take corrective actions now Troubleshoot
faster and improve end-user experience. Signup Now! http://puba
ds.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140__________
_____________________________________ qooxdoo-devel mailing
list qooxdoo-
listinfo/qooxdoo-devel
-------------------------------------------------------------
-----------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just
$35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/414
0
_______________________________________________
qooxdoo-devel mailing list
t/lists/listinfo/qooxdoo-devel
---------------------------------------------------------------
--------------- Site24x7 APM Insight: Get Deep Visibility into
Application Performance APM + Mobile APM + RUM: Monitor 3 App
instances at just $35/Month Monitor end-to-end web transactions
and take corrective actions now Troubleshoot faster and improve
end-user experience. Signup Now! http://pubads.g.doubleclick.net/
gampad/clk?id=272487151&iu=/4140_________________________________
______________ qooxdoo-devel mailing list qooxdoo-
stinfo/qooxdoo-devel
---------------------------------------------------------------
---------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
lists/listinfo/qooxdoo-devel
-----------------------------------------------------------------
------------- Site24x7 APM Insight: Get Deep Visibility into
Application Performance APM + Mobile APM + RUM: Monitor 3 App
instances at just $35/Month Monitor end-to-end web transactions and
take corrective actions now Troubleshoot faster and improve end-
user experience. Signup Now! http://pubads.g.doubleclick.net/gampad
/clk?id=272487151&iu=/4140_________________________________________
______ qooxdoo-devel mailing list qooxdoo-
info/qooxdoo-devel
-----------------------------------------------------------------
-------------
Site24x7 APM Insight: Get Deep Visibility into Application
Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
sts/listinfo/qooxdoo-devel
-------------------------------------------------------------------
----------- Site24x7 APM Insight: Get Deep Visibility into
Application Performance APM + Mobile APM + RUM: Monitor 3 App
instances at just $35/Month Monitor end-to-end web transactions and
take corrective actions now Troubleshoot faster and improve end-user
experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id
=272487151&iu=/4140_______________________________________________
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
-------------------------------------------------------------------
-----------
Site24x7 APM Insight: Get Deep Visibility into Application
Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
Dimitri
2016-02-21 20:40:15 UTC
Permalink
John,
Seems like our projects have slightly different scope, but at the same
time they'll definitely have much in common. We currently focus on
JavaEE for our backend development, and radical paradigm change is
unlikely in foreseeable future.
The key points of the forthcoming project will be:
- JSR-223 support, with Nashorn in mind first;
- CDI injection for scripts: @Inject, @Persistense{Context,Unit},
@Resource, @EJB;
- script execution: programmatic, manual/interactive, scheduled, on CDI
event;
- interfaces: Java API, REST, GUI (a la Playground, most probably also
qooxdoo+Ace.js based);
- script libraries: bundled with application (r/o) and user (r/w),
managed with the same APIs.
Problems solved:
- experiment with real-world data, run JPQL queries live, make
development more agile;
- implement fluid logic pattern, adjust business logic without
recompilation and redeployment;
- do maintenance tasks and some simple analytic in real time.
As you see, this will be mainly a productivity tool for JavaEE
developers, while your development (if I got it right) presumes
completely new qooxdoo-with-Node-on-top-of-Nashorn solution.
However, I can already see some common points. Seems like CDI
injections could be topical for you, too; at the same time, it's the
most non-trivial tasks of the above. I would gladly continue the
discussion, but I'm afraid we'll further pollute the ML. In particular,
I wanted to know your opinion (as an experienced server-side JS user)
on how to attach metadata to scripts (external file or embedded @-
annotations).
Dimitri
P.S. Have you heard of WildFly's Undertow.js project? It's not exactly
what either you or me are looking for, but we can borrow some ideas
from it.
Slightly offtopic: did you try Nashorn with your server-side?
Should be
much faster and feature-rich. Besides, we are planning an
opensource
project - JavaScript console for JavaEE with qooxdoo frontend a la
Playground. It would allow to author/run/manage server-side
scripts,
with support for CDI injections, scheduled execution, user/system
script library etc. I'd like to ask you (and everyone else here) to
let
me know if you're interested.
Yes, I am - I want to move my server platform in that direction, so
that I can modify code on the production server on the fly, but it’s
quite a large development and not going to be capable of being split
off into open source any time soon.  
Long term I want to move to more javascript on the server, and this
could be node.js based but doing that would require a complete
redevelopment of my server architecture; so an alternative might be
to switch to Nashorn and run the node.js API on top of Nashorn.  This
would be particularly cool if that meant that the core platform could
run either in native node.js or be embedded with Java, because then
there are two major types of users the project would appeal to.
IMHO there are a lots of benefits to this approach, eg having the
Javascript architecture based on Qooxdoo will bring OO maturity that
Java devs will appreciate, JS devs will be able to do end-to-end JS
in ES6, and JavaEE devs will be able to add rapid scripting to their
enterprise platform :)  Not to mention that asynchronous everything
is an extremely scalable architecture, something that Java has
traditionally been poor on.
One of the limitations that QxCompiler will solve is high speed
incremental compilations and the ability to detect when the server
needs to reload it’s VM so it’s a deliberate first step in this
direction.
What do you think?  Lets keep chatting about it, how far along are
your plans?
John
-------------------------------------------------------------------
-----------
Site24x7 APM Insight: Get Deep Visibility into Application
Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
Dietrich Streifert
2016-02-15 08:04:53 UTC
Permalink
Hi John

Congratulations!
*NOTE*While QxCompiler is backward compatible with generate.py and
does not require changes to your code, ES6 support requires that
strict mode is enabled and this can introduce a few minor
compatibility issues; also, QxCompiler uses a new trick for managing
dependencies and for both of these reasons you must use my fork of
Qooxdoo (https://github.com/johnspackman/qooxdooand use
the*qxcompiler*branch). My fork is based on Qooxdoo 5.0.1.
Could you explain which changes where necessary in qooxdoo to make it
work with your compiler? Would it maybe be possible to "document" those
changes via a pull request against qooxdoo/qooxdoo?

Another question is related to browser compatibilities: are they
changed/raised because of the ES6 and strict mode changes? Will it still
run on IE8?

Regards
Dietrich
John Spackman
2016-02-15 08:28:20 UTC
Permalink
Hi Dietrich

Thanks :)

There are two sets of changes – strict mode support (which is in the strict-mode-support branch), and mods needed for QxCompiler. I will put them up as pull requests once things settle down with QxCompiler, but in the mean time both branches are based on vanilla Qoxodoo 5.0.1 (ie not including any of my other branches) so there shouldn’t be a barrier for anyone to use my branch.

Strict mode fixes are:
(a) You cannot write to properties which do not have a getter; in various places Qooxdoo will copy event properties back to the event object (which AFAICT is a fix for IE8) but this raises an exception in strict mode so needs a try/catch wrapper
(b) The global object is undefined, not “window” so if you want a global object (like “q”) it needs the “window” qualifier

As for IE8 support – I’ve not tested it but AIUI strict mode will not cause issues in IE8/IE9, and the changes I’ve made are backward compatible.

The QxCompiler fixes are to do with dependencies – basically, the load dependencies of a Qooxdoo app are greatly complicated because classes can have a defer() method, which allows code to be run before the app is fully loaded. The way (I think/guess) that the generate.py does it is to recursively interpret the code in .defer() and all of the methods it calls, ie it tries to predict at compile-time what methods will be called at runtime so that it can make sure that the load order is correct. As you can imagine this is non-trivial, but IMHO this makes it easy for minor code changes to have a big impact on load order and to cause unresolvable recursive dependency issues.

My solution is to not call the class .defer() method until all classes are loaded (that’s not strictly possible because some classes .defer() must be called, but the dependencies are a lot simpler). This solution needs a a couple of backwards compatible mods, mostly in qx.Bootstrap, and these are in the qxcompiler branch

There’s bit more detail here: https://github.com/johnspackman/qxcompiler/blob/master/docs/Dependencies.md

Cheers
John

From: Dietrich Streifert <***@googlemail.com>
Reply-To: qooxdoo Development <qooxdoo-***@lists.sourceforge.net>
Date: Monday, 15 February 2016 at 08:04
To: qooxdoo Development <qooxdoo-***@lists.sourceforge.net>
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster compilation, and 100% Javascript API to building applications


Hi John

Congratulations!

One question regarding this passage in the README.md:


NOTE While QxCompiler is backward compatible with generate.py and does not require changes to your code, ES6 support requires that strict mode is enabled and this can introduce a few minor compatibility issues; also, QxCompiler uses a new trick for managing dependencies and for both of these reasons you must use my fork of Qooxdoo (https://github.com/johnspackman/qooxdoo and use the qxcompiler branch). My fork is based on Qooxdoo 5.0.1.

Could you explain which changes where necessary in qooxdoo to make it work with your compiler? Would it maybe be possible to "document" those changes via a pull request against qooxdoo/qooxdoo?

Another question is related to browser compatibilities: are they changed/raised because of the ES6 and strict mode changes? Will it still run on IE8?

Regards
Dietrich



------------------------------------------------------------------------------ Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140_______________________________________________ qooxdoo-devel mailing list qooxdoo-***@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
thron7
2016-02-15 22:34:06 UTC
Permalink
John,

this looks interesting! I like the API-based approach (Reminds me of the
Clojure "boot" build system's tag line, "Builds are programs"[1]). It seems
you have been working on that for a while.

[1] http://boot-clj.com/
Post by John Spackman
The QxCompiler fixes are to do with dependencies – basically, the load
dependencies of a Qooxdoo app are greatly complicated because classes can
have a defer() method, which allows code to be run before the app is fully
loaded. The way (I think/guess) that the generate.py does it is to
recursively interpret the code in .defer() and all of the methods it calls,
ie it tries to predict at compile-time what methods will be called at
runtime so that it can make sure that the load order is correct. As you
can imagine this is non-trivial, but IMHO this makes it easy for minor code
changes to have a big impact on load order and to cause unresolvable
recursive dependency issues.
My solution is to not call the class .defer() method until all classes are
loaded (that’s not strictly possible because some classes .defer() must be
called, but the dependencies are a lot simpler). This solution needs a a
couple of backwards compatible mods, mostly in qx.Bootstrap, and these are
in the qxcompiler branch
https://github.com/johnspackman/qxcompiler/blob/master/docs/Dependencies.md
What really complicates dependency inference is to find the transitive
closure for each (what you put as "recursively interpret the code"). But I
think you need that for both load-time and run-time dependencies alike.
This entails that any small change in far away code can have an impact on
the overall set of classes and their load order. (But only load-time makes
cyclic dependencies an issue, and that's probably what you care about).

Once recursive analysis is in place, adding the symbols from .defer() to
the load-time dependencies is easy as you write. So it's not that .defer()
makes the general mechanism of treating dependencies more difficult - it
just enlarges the set of load-time dependencies, and hence increases the
risk of not being able to create a partial order for all the classes. But
.defer() is not particularly more problematic than, say, static
initializers in the class, or #require's.

But I see how .defer() is a good target to minimize on that risk.

T.
John Spackman
2016-02-16 06:47:28 UTC
Permalink
Thanks Thomas :)

I like the API approach too, I think it opens up some possibilities (I remember you had it on your todo list for a while and I can see why). Its definitely been a while coming, I wrote the proof of concept years ago with Esprima but mortgage-paying work always took priority!

With my approach to dependencies, QxCompiler is taking a shortcut but reducing that target appears to be very profitable; there have been a couple of cases where I’ve had to add in @require to Qooxdoo classes, this is typically where (e.g.) a qx.core.Environment provider class uses a static method to initialise instead of directly in .defer() and there is an additional dependency, but there’s a lot of cases where explicit @require() is already present so my mods to the framework have been kept to a minimum.

I quite like that the database (the equivalent of generator’s cache) is kept quite small too - around 1Mb, and tracking the dependencies of methods would add a lot of data as well as code complexity so if I can keep it this way then that would be ideal.

Regards
John

From: thron7 <***@users.sourceforge.net>
Reply-To: qooxdoo Development <qooxdoo-***@lists.sourceforge.net>
Date: Monday, 15 February 2016 at 22:34
To: qooxdoo Development <qooxdoo-***@lists.sourceforge.net>
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster compilation, and 100% Javascript API to building applications

John,

this looks interesting! I like the API-based approach (Reminds me of the Clojure "boot" build system's tag line, "Builds are programs"[1]). It seems you have been working on that for a while.

[1] http://boot-clj.com/

On Mon, Feb 15, 2016 at 9:28 AM, John Spackman <john-***@zenesis.com> wrote:


The QxCompiler fixes are to do with dependencies – basically, the load dependencies of a Qooxdoo app are greatly complicated because classes can have a defer() method, which allows code to be run before the app is fully loaded. The way (I think/guess) that the generate.py does it is to recursively interpret the code in .defer() and all of the methods it calls, ie it tries to predict at compile-time what methods will be called at runtime so that it can make sure that the load order is correct. As you can imagine this is non-trivial, but IMHO this makes it easy for minor code changes to have a big impact on load order and to cause unresolvable recursive dependency issues.

My solution is to not call the class .defer() method until all classes are loaded (that’s not strictly possible because some classes .defer() must be called, but the dependencies are a lot simpler). This solution needs a a couple of backwards compatible mods, mostly in qx.Bootstrap, and these are in the qxcompiler branch

There’s bit more detail here: https://github.com/johnspackman/qxcompiler/blob/master/docs/Dependencies.md

What really complicates dependency inference is to find the transitive closure for each (what you put as "recursively interpret the code"). But I think you need that for both load-time and run-time dependencies alike. This entails that any small change in far away code can have an impact on the overall set of classes and their load order. (But only load-time makes cyclic dependencies an issue, and that's probably what you care about).

Once recursive analysis is in place, adding the symbols from .defer() to the load-time dependencies is easy as you write. So it's not that .defer() makes the general mechanism of treating dependencies more difficult - it just enlarges the set of load-time dependencies, and hence increases the risk of not being able to create a partial order for all the classes. But .defer() is not particularly more problematic than, say, static initializers in the class, or #require's.

But I see how .defer() is a good target to minimize on that risk.

T.
------------------------------------------------------------------------------ Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140_______________________________________________ qooxdoo-devel mailing list qooxdoo-***@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
Defero
2016-02-15 17:15:10 UTC
Permalink
Hi,

congratulations on the release

I've already started to play around a bit and will send you a pull request
for it. While trying to set it up, i've run into a little snag at the
following:

qxcompiler\lib\util.js
line: 305
code : return fs.mkdir(made, function(err) {

and because fs.mkdir (in my version of node at least) doesnt create folders
recursively, it failed the compilation of the compile-app-demo.js. I changed
to the mkdirp module.

this is looking sweet though :)



--
View this message in context: http://qooxdoo.678.n2.nabble.com/QxCompiler-add-ES6-faster-compilation-and-100-Javascript-API-to-building-applications-tp7587992p7588003.html
Sent from the qooxdoo mailing list archive at Nabble.com.
John Spackman
2016-02-16 06:52:18 UTC
Permalink
Hi Defero

Thanks :)

I found a bug in util.mkpath yesterday that might have been the problem you found - mkpath would start to create the folders and then stop, but that’s fixed now.

Are you thinking of having a look at part loading?

John
Post by Defero
Hi,
congratulations on the release
I've already started to play around a bit and will send you a pull request
for it. While trying to set it up, i've run into a little snag at the
qxcompiler\lib\util.js
line: 305
code : return fs.mkdir(made, function(err) {
and because fs.mkdir (in my version of node at least) doesnt create folders
recursively, it failed the compilation of the compile-app-demo.js. I changed
to the mkdirp module.
this is looking sweet though :)
--
View this message in context: http://qooxdoo.678.n2.nabble.com/QxCompiler-add-ES6-faster-compilation-and-100-Javascript-API-to-building-applications-tp7587992p7588003.html
Sent from the qooxdoo mailing list archive at Nabble.com.
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
Defero
2016-02-16 10:47:12 UTC
Permalink
Hi,

currently i'm still in the process of playing around with it, looking under
the hood a bit in my free time But yeah, it will be on my priority todo list
to create a part loader.


Defero



--
View this message in context: http://qooxdoo.678.n2.nabble.com/QxCompiler-add-ES6-faster-compilation-and-100-Javascript-API-to-building-applications-tp7587992p7588011.html
Sent from the qooxdoo mailing list archive at Nabble.com.
Defero
2016-02-25 13:13:36 UTC
Permalink
Hi John,

so i've been thinking how i would start creating the part loader which
should be "easy" to create with how you set up the things.
But for it to fit our purpose here, it asks that only a single part/library
can be recompiled.

I'm not sure if there is a nice way of achieving this. In case of two parts
have the same dependency but are not aware of each other. Which in most UI
qx classes i think shouldn't be an issue if they are compiled/loaded twice.
But i'm sure there are plenty of classes that have to strictly be loaded
once onto dom.

Do you think there is a way to achieve this without cross-referencing every
part and adding to boot the same dependencies.

Regards,
Defero



--
View this message in context: http://qooxdoo.678.n2.nabble.com/QxCompiler-add-ES6-faster-compilation-and-100-Javascript-API-to-building-applications-tp7587992p7588112.html
Sent from the qooxdoo mailing list archive at Nabble.com.
John Spackman
2016-02-27 21:27:11 UTC
Permalink
Hi Defero

Sorry for the delay in getting back to you, it's been a very busy week

I've not used parts in Qooxdoo, but AIUI you associate classes with a
named part, and then use the API at runtime to force the loading of the
part; how the compiler brings together for a part would be the same as for
a "normal" application in that it starts with a list of classes and finds
their dependencies. The difference with parts would be that the list of
classes in a part would exclude anything already loaded.

Given that a build target collects multiple classes into a few files, then
either (a) each class can exist in exactly one file, or (b) the merged
files have to detect whether a class already exists and not call
qx.Class.define(). Obviously option (a) means the developer has to have a
more detailed knowledge of dependencies, but (b) would make life a lot
easier.

Given also that which parts get loaded in what order would depend on what
a user does, it seems to me that (b) may even be a requirement? I've not
thought this through yet, but I'm wondering if you can analyse the
overlapping sets so that for, say, 5 parts you might end up with say, 20
files - ie broken down into enough granularity that when you request a
specific part, and the partloader knows what parts are already loaded,
there is a specific set of files to load without there being the
possibility of duplicate classes.

Im not sure if that helps, but it might be good to start with looking at
what generate.py does now? Lets keep the conversation going though, I'd
happily talk through ideas :)

Cheers
John


----------------------------------------
From: "Defero" <***@gmail.com>
Sent: Thursday, February 25, 2016 1:26 PM
To: qooxdoo-***@lists.sourceforge.net
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster compilation, and
100% Javascript API to building applications
Hi John,

so i've been thinking how i would start creating the part loader which
should be "easy" to create with how you set up the things.
But for it to fit our purpose here, it asks that only a single
part/library
can be recompiled.

I'm not sure if there is a nice way of achieving this. In case of two
parts
have the same dependency but are not aware of each other. Which in most UI
qx classes i think shouldn't be an issue if they are compiled/loaded
twice.
But i'm sure there are plenty of classes that have to strictly be loaded
once onto dom.

Do you think there is a way to achieve this without cross-referencing
every
part and adding to boot the same dependencies.

Regards,
Defero

--
View this message in context:
http://qooxdoo.678.n2.nabble.com/QxCompiler-add-ES6-faster-compilation-and-1
00-Javascript-API-to-building-applications-tp7587992p7588112.html
Sent from the qooxdoo mailing list archive at Nabble.com.

----------------------------------------------------------------------------
--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140

d***@cost-savers.net
2016-02-16 11:25:48 UTC
Permalink
Congratulations John!

This is the major step taken the last 18 months!!! (cause nothing much happens with qooxdoo anymore while ExtJS and other frameworks develop quite fast)
You have proven it come be done in a very delicate way.
You give the core team a huge challenge...the question is if they can handle it the right way. No proven good track record in this area...;-(
Anyways, we have been testing it and it looks amazing and I am sure all your work can be reused by all of us to increase modularity...

Thanks!

Stefan
Post by John Spackman
Thanks Thomas :)
I like the API approach too, I think it opens up some possibilities (I remember you had it on your todo list for a while and I can see why). Its definitely been a while coming, I wrote the proof of concept years ago with Esprima but mortgage-paying work always took priority!
I quite like that the database (the equivalent of generator’s cache) is kept quite small too - around 1Mb, and tracking the dependencies of methods would add a lot of data as well as code complexity so if I can keep it this way then that would be ideal.
Regards
John
Date: Monday, 15 February 2016 at 22:34
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster compilation, and 100% Javascript API to building applications
John,
this looks interesting! I like the API-based approach (Reminds me of the Clojure "boot" build system's tag line, "Builds are programs"[1]). It seems you have been working on that for a while.
[1] http://boot-clj.com/
The QxCompiler fixes are to do with dependencies – basically, the load dependencies of a Qooxdoo app are greatly complicated because classes can have a defer() method, which allows code to be run before the app is fully loaded. The way (I think/guess) that the generate.py does it is to recursively interpret the code in .defer() and all of the methods it calls, ie it tries to predict at compile-time what methods will be called at runtime so that it can make sure that the load order is correct. As you can imagine this is non-trivial, but IMHO this makes it easy for minor code changes to have a big impact on load order and to cause unresolvable recursive dependency issues.
My solution is to not call the class .defer() method until all classes are loaded (that’s not strictly possible because some classes .defer() must be called, but the dependencies are a lot simpler). This solution needs a a couple of backwards compatible mods, mostly in qx.Bootstrap, and these are in the qxcompiler branch
There’s bit more detail here: https://github.com/johnspackman/qxcompiler/blob/master/docs/Dependencies.md
What really complicates dependency inference is to find the transitive closure for each (what you put as "recursively interpret the code"). But I think you need that for both load-time and run-time dependencies alike. This entails that any small change in far away code can have an impact on the overall set of classes and their load order. (But only load-time makes cyclic dependencies an issue, and that's probably what you care about).
Once recursive analysis is in place, adding the symbols from .defer() to the load-time dependencies is easy as you write. So it's not that .defer() makes the general mechanism of treating dependencies more difficult - it just enlarges the set of load-time dependencies, and hence increases the risk of not being able to create a partial order for all the classes. But .defer() is not particularly more problematic than, say, static initializers in the class, or #require's.
But I see how .defer() is a good target to minimize on that risk.
T.
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
John Spackman
2016-02-16 14:28:27 UTC
Permalink
Hi Stefan

Glad you like it :) I think that the important thing about a VCS like git is that it is easier to take things forward ourselves if we want to. So my qxcompiler branch is designed to easily fit in with anyone elses. 1&1 have their own corporate direction which may not have much focus on Qooxdoo any more, and although there might be a connection between the MIT license and decentralising the core repo I haven’t heard anything yet

But I’d be interested to collaborate on an official community fork; there may not be a huge explosion of development activity ;) but it would give us (the community) the opportunity to take the project forward, merging in changes by the core 1&1 team.

What do you/anyone think?

John
Post by d***@cost-savers.net
Congratulations John!
This is the major step taken the last 18 months!!! (cause nothing much happens with qooxdoo anymore while ExtJS and other frameworks develop quite fast)
You have proven it come be done in a very delicate way.
You give the core team a huge challenge...the question is if they can handle it the right way. No proven good track record in this area...;-(
Anyways, we have been testing it and it looks amazing and I am sure all your work can be reused by all of us to increase modularity...
Thanks!
Stefan
Post by John Spackman
Thanks Thomas :)
I like the API approach too, I think it opens up some possibilities (I remember you had it on your todo list for a while and I can see why). Its definitely been a while coming, I wrote the proof of concept years ago with Esprima but mortgage-paying work always took priority!
I quite like that the database (the equivalent of generator’s cache) is kept quite small too - around 1Mb, and tracking the dependencies of methods would add a lot of data as well as code complexity so if I can keep it this way then that would be ideal.
Regards
John
Date: Monday, 15 February 2016 at 22:34
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster compilation, and 100% Javascript API to building applications
John,
this looks interesting! I like the API-based approach (Reminds me of the Clojure "boot" build system's tag line, "Builds are programs"[1]). It seems you have been working on that for a while.
[1] http://boot-clj.com/
The QxCompiler fixes are to do with dependencies – basically, the load dependencies of a Qooxdoo app are greatly complicated because classes can have a defer() method, which allows code to be run before the app is fully loaded. The way (I think/guess) that the generate.py does it is to recursively interpret the code in .defer() and all of the methods it calls, ie it tries to predict at compile-time what methods will be called at runtime so that it can make sure that the load order is correct. As you can imagine this is non-trivial, but IMHO this makes it easy for minor code changes to have a big impact on load order and to cause unresolvable recursive dependency issues.
My solution is to not call the class .defer() method until all classes are loaded (that’s not strictly possible because some classes .defer() must be called, but the dependencies are a lot simpler). This solution needs a a couple of backwards compatible mods, mostly in qx.Bootstrap, and these are in the qxcompiler branch
There’s bit more detail here: https://github.com/johnspackman/qxcompiler/blob/master/docs/Dependencies.md
What really complicates dependency inference is to find the transitive closure for each (what you put as "recursively interpret the code"). But I think you need that for both load-time and run-time dependencies alike. This entails that any small change in far away code can have an impact on the overall set of classes and their load order. (But only load-time makes cyclic dependencies an issue, and that's probably what you care about).
Once recursive analysis is in place, adding the symbols from .defer() to the load-time dependencies is easy as you write. So it's not that .defer() makes the general mechanism of treating dependencies more difficult - it just enlarges the set of load-time dependencies, and hence increases the risk of not being able to create a partial order for all the classes. But .defer() is not particularly more problematic than, say, static initializers in the class, or #require's.
But I see how .defer() is a good target to minimize on that risk.
T.
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
d***@cost-savers.net
2016-02-16 15:21:24 UTC
Permalink
Hi,
Post by John Spackman
But I’d be interested to collaborate on an official community fork; there may not be a huge explosion of development activity ;) but it would give us (the community) the opportunity to take the project forward, merging in changes by the core 1&1 team.
What do you/anyone think?
We would definitively be part of it as we have invested too much in it to leave.
The goal should not be set too high at least in the start.
Instead try to focus on some necessary improvements and take it from there.
You are already on the road in that direction...

It would be good cause then there will be three roads to go:
1. the currently somnolent 1&1 road
2. the at-least-something-happen-community-road
3. the joint-road where 1&1 opens up, as they should have done long ago....

I prefer alt 3 but can as well choose the alt 2.

Stefan
Post by John Spackman
John
Post by d***@cost-savers.net
Congratulations John!
This is the major step taken the last 18 months!!! (cause nothing much happens with qooxdoo anymore while ExtJS and other frameworks develop quite fast)
You have proven it come be done in a very delicate way.
You give the core team a huge challenge...the question is if they can handle it the right way. No proven good track record in this area...;-(
Anyways, we have been testing it and it looks amazing and I am sure all your work can be reused by all of us to increase modularity...
Thanks!
Stefan
Post by John Spackman
Thanks Thomas :)
I like the API approach too, I think it opens up some possibilities (I remember you had it on your todo list for a while and I can see why). Its definitely been a while coming, I wrote the proof of concept years ago with Esprima but mortgage-paying work always took priority!
I quite like that the database (the equivalent of generator’s cache) is kept quite small too - around 1Mb, and tracking the dependencies of methods would add a lot of data as well as code complexity so if I can keep it this way then that would be ideal.
Regards
John
Date: Monday, 15 February 2016 at 22:34
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster compilation, and 100% Javascript API to building applications
John,
this looks interesting! I like the API-based approach (Reminds me of the Clojure "boot" build system's tag line, "Builds are programs"[1]). It seems you have been working on that for a while.
[1] http://boot-clj.com/
The QxCompiler fixes are to do with dependencies – basically, the load dependencies of a Qooxdoo app are greatly complicated because classes can have a defer() method, which allows code to be run before the app is fully loaded. The way (I think/guess) that the generate.py does it is to recursively interpret the code in .defer() and all of the methods it calls, ie it tries to predict at compile-time what methods will be called at runtime so that it can make sure that the load order is correct. As you can imagine this is non-trivial, but IMHO this makes it easy for minor code changes to have a big impact on load order and to cause unresolvable recursive dependency issues.
My solution is to not call the class .defer() method until all classes are loaded (that’s not strictly possible because some classes .defer() must be called, but the dependencies are a lot simpler). This solution needs a a couple of backwards compatible mods, mostly in qx.Bootstrap, and these are in the qxcompiler branch
There’s bit more detail here: https://github.com/johnspackman/qxcompiler/blob/master/docs/Dependencies.md
What really complicates dependency inference is to find the transitive closure for each (what you put as "recursively interpret the code"). But I think you need that for both load-time and run-time dependencies alike. This entails that any small change in far away code can have an impact on the overall set of classes and their load order. (But only load-time makes cyclic dependencies an issue, and that's probably what you care about).
Once recursive analysis is in place, adding the symbols from .defer() to the load-time dependencies is easy as you write. So it's not that .defer() makes the general mechanism of treating dependencies more difficult - it just enlarges the set of load-time dependencies, and hence increases the risk of not being able to create a partial order for all the classes. But .defer() is not particularly more problematic than, say, static initializers in the class, or #require's.
But I see how .defer() is a good target to minimize on that risk.
T.
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
Dimitri
2016-02-16 22:34:16 UTC
Permalink
Hi everyone,

I second every word: there's too much invested. After all, qooxdoo is
too cool to let it become abandonware :)

Be it 2nd or 3rd road, I'm in.

Dimitri
Post by d***@cost-savers.net
Hi,
Post by John Spackman
But I’d be interested to collaborate on an official community fork;
there may not be a huge explosion of development activity ;) but it
would give us (the community) the opportunity to take the project
forward, merging in changes by the core 1&1 team.
What do you/anyone think?
We would definitively be part of it as we have invested too much in it to leave.
The goal should not be set too high at least in the start.
Instead try to focus on some necessary improvements and take it from there.
You are already on the road in that direction...
1. the currently somnolent 1&1 road
2. the at-least-something-happen-community-road
3. the joint-road where 1&1 opens up, as they should have done long ago....
I prefer alt 3 but can as well choose the alt 2.
Stefan
Post by John Spackman
John
Post by d***@cost-savers.net
Congratulations John!
This is the major step taken the last 18 months!!! (cause nothing
much happens with qooxdoo anymore while ExtJS and other
frameworks develop quite fast)
You have proven it come be done in a very delicate way.
You give the core team a huge challenge...the question is if they
can handle it the right way. No proven good track record in this
area...;-(
Anyways, we have been testing it and it looks amazing and I am
sure all your work can be reused by all of us to increase
modularity...
Thanks!
Stefan
Post by John Spackman
Thanks Thomas :)
I like the API approach too, I think it opens up some
possibilities (I remember you had it on your todo list for a
while and I can see why).  Its definitely been a while coming,
I wrote the proof of concept years ago with Esprima but
mortgage-paying work always took priority!
With my approach to dependencies, QxCompiler is taking a
shortcut but reducing that target appears to be very
profitable; there have been a couple of cases where I’ve had to
(e.g.) a qx.core.Environment provider class uses a static
method to initialise instead of directly in .defer() and there
is an additional dependency, but there’s a lot of cases where
framework have been kept to a minimum.
I quite like that the database (the equivalent of generator’s
cache) is kept quite small too - around 1Mb, and tracking the
dependencies of methods would add a lot of data as well as code
complexity so if I can keep it this way then that would be
ideal.
Regards
John
.net>
Date:  Monday, 15 February 2016 at 22:34
Subject:  Re: [qooxdoo-devel] QxCompiler - add ES6, faster
compilation, and 100% Javascript API to building applications
John,
this looks interesting! I like the API-based approach (Reminds
me of the Clojure "boot" build system's tag line, "Builds are
programs"[1]). It seems you have been working on that for a
while.
[1] http://boot-clj.com/
The QxCompiler fixes are to do with dependencies – basically,
the load dependencies of a Qooxdoo app are greatly complicated
because classes can have a defer() method, which allows code to
be run before the app is fully loaded.  The way (I think/guess)
that the generate.py does it is to recursively interpret the
code in .defer() and all of the methods it calls, ie it tries
to predict at compile-time what methods will be called at
runtime so that it can make sure that the load order is
correct.  As you can imagine this is non-trivial, but IMHO this
makes it easy for minor code changes to have a big impact on
load order and to cause unresolvable recursive dependency
issues.
My solution is to not call the class .defer() method until all
classes are loaded (that’s not strictly possible because some
classes .defer() must be called, but the dependencies are a lot
simpler).  This solution needs a a couple of backwards
compatible mods, mostly in qx.Bootstrap, and these are in the
qxcompiler branch
There’s bit more detail here: https://github.com/johnspackman/q
xcompiler/blob/master/docs/Dependencies.md
What really complicates dependency inference is to find the
transitive closure for each (what you put as "recursively
interpret the code"). But I think you need that for both load-
time and run-time dependencies alike. This entails that any
small change in far away code can have an impact on the overall
set of classes and their load order. (But only load-time makes
cyclic dependencies an issue, and that's probably what you care
about).
Once recursive analysis is in place, adding the symbols from
.defer() to the load-time dependencies is easy as you write. So
it's not that .defer() makes the general mechanism of treating
dependencies more difficult - it just enlarges the set of load-
time dependencies, and hence increases the risk of not being
able to create a partial order for all the classes. But
.defer() is not particularly more problematic than, say, static
initializers in the class, or #require's. 
But I see how .defer() is a good target to minimize on that risk.
T.
-------------------------------------------------------------
----------------- Site24x7 APM Insight: Get Deep Visibility
into Application Performance APM + Mobile APM + RUM: Monitor 3
App instances at just $35/Month Monitor end-to-end web
transactions and take corrective actions now Troubleshoot
faster and improve end-user experience. Signup Now! http://puba
ds.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140__________
_____________________________________ qooxdoo-devel mailing
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
-------------------------------------------------------------
-----------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just
$35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/414
0
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
---------------------------------------------------------------
---------------
Site24x7 APM Insight: Get Deep Visibility into Application
Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
-----------------------------------------------------------------
-------------
Site24x7 APM Insight: Get Deep Visibility into Application
Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
-------------------------------------------------------------------
-----------
Site24x7 APM Insight: Get Deep Visibility into Application
Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
Dimitri
2016-02-21 19:23:50 UTC
Permalink
John, Fritz,

Yes exactly - I was talking about profiling QxCompiler itself. At that
moment I was experimenting with shared database and was wondering why
it doesn't bring any speedup. I thought profiling would help to
understand where QxCompiler spends most time; later it turned to be
more fundamental issue.

I've experimented with a bit with incremental builds in our CI. So far,
I think incremental builds are fine for CI builds (triggered by
commits). That build directory isn't cleared each time didn't cause any
complications yet. It's not a problem that first build takes
significant time, it's only time of subsequent builds that matters.
It's also a good practice to perform clean builds for releases, but
slow release builds are quite acceptable since they don't occur
frequently.

However, I don't think efficient shared cache is impossible "by
design". I've looked into {qxt,myproject}/{source,build}-
output/transpiled/qx/*, did a byte-by-byte comparison and found that
*.js files are identical (therefore, they could probably be cached
across targets and libraries). Of course, there was difference in which
files were present, due to different dependencies.

(Besides, all the *.map files contain "null", are they really needed?)

Speaking of dependencies, I did some comparison between qxt and our
project:

Framework classes used: 283 vs. 378
Contribs: UploadMgr (both)
Uncompressed script: 4.8M vs. 6.4M
Compressed script (QxCompiler): 1.4M vs. 1.9M
Compressed script (generate.py): 700K vs. 1M

Definitely there is some room for optimization :)

John, sorry for delay in conversation. I'll be answering you on server-
side JS soon.

Dimitri
Hi Fritz
 
In that case that's easy ;) because there's no profiling info in
QxCompiler at the moment, although I do keep an eye on overall
compilation time; the two main costs are transpiling, because every
source file in every library has to be transpiled, and the other is
copying resources for the build target.  
 
It can be difficult to gauge timings without isolating parts of the
app, although my guess is that a lot of this is probably file I/O;
the actual Babel transpilation process is a notable exception in that
it's completely synchronous and completely CPU bound
 
Initially, I wanted to keep the transpiled code local to each library
but that isn't possible because some targets affect the output of the
transpile stage, but that just means that it has to be cached by
target rather than by library.
 
There's always room for improvement but overall I think it's quite
fast given the work that it's doing, and the incremental compile for
source targets is around 800ms on my laptop, but build target needs a
bit of a once over because it around 6secs and it should be a lot
faster if nothing's changed.
 
Cheers
John
 
 
Sent: Friday, February 19, 2016 8:42 AM
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster
compilation, and 100% Javascript API to building applications
 
Hi John,
I understood it as a means to profile QxCompiler. But even if Dimitri
had
something else in mind, it probably would be useful to know where
QxCompiler
spends most of it's time in order to focus on optimizing at the right
place.
BWT, I am looking forward to trying QxCompiler myself, just didn't
get
around to it yet.
Cheers,
Fritz
BTW I missed you comment on this earlier - by profiling, do you
mean of
the compilation process or injecting profiling statements into the
generated code?
I'm not a Node.js pro (I'm a JavaEE architect in fact :) so I'm
not
experienced in profiling Node applications. It would be nice if
some
verbose output (with timings) was produced by QxCompiler, as a
poor
man's profiling facility :)
-----------------------------------------------------------------
-------------
Site24x7 APM Insight: Get Deep Visibility into Application
Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
--
Oetiker+Partner AG tel: +41 62 775 9903 (direct)
Fritz Zaucker +41 62 775 9900 (switch board)
Aarweg 15 +41 79 675 0630 (mobile)
CH-4600 Olten fax: +41 62 775 9905
Schweiz web: www.oetiker.ch
-------------------------------------------------------------------
-----------
Site24x7 APM Insight: Get Deep Visibility into Application
Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
 
 ------------------------------------------------------------------
------------
Site24x7 APM Insight: Get Deep Visibility into Application
Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
John Spackman
2016-02-22 18:49:10 UTC
Permalink
There’s a new release that went out just before the translation mod that does the shared cache - although the source-output directory is currently very similar to build-output, this is only temporary because build options can change the code which is output.

This is primarily that calls to qx.core.Environment.get and .select are optimised out where QxCompiler or generate.py can know the outcome; for example, in build target where the qx.debug is set to false, QxCompiler can remove this code completely:
if (qx.core.Environment.get("qx.debug")) {
// ... snip ...
}
I think that this is a major difference in size between ./generate.py build target and the QxCompiler, because QxCompiler currently does not do the elimination.

NOTE: I have renamed qxcompiler.makers.SimpleApp to qxcompiler.makers.AppMaker to follow other naming conventions.

So in the changes I pushed earlier this evening, the AppMaker now supports making multiple Applications simultaneously – you can see an example in ./test/compile-app-demo.js and ./test/compile-app-demo-build.js:

maker.addApplication(new qxcompiler.Application("qxt.Application").set({
theme: "qxt.theme.Theme",
name: "appone",
environment: {
"qxt.applicationName": "App One"
}
}));
maker.addApplication(new qxcompiler.Application("qxt.Application").set({
theme: "qx.theme.Simple",
name: "apptwo",
environment: {
"qxt.applicationName": "App Two"
}
}));

For every Application you want to compile, just add a new instance of qxcompiler.Application; the first parameter to the constructor is the main class name for the application. You specify the theme as a property, and note the new “name” property – previously, one application related to an “index.html” file that loaded files from the “script” folder, whereas in this new release the name property is used to name both the .html file and the script folder, so each application needs a unique name.

In the example above, the two applications differ just in theme but because they share the same output directory, the overhead of second application adds around 400ms on my laptop when compared to just the one application.

This might not be exactly what you were expecting when compared to generate.py because entire applications can be sent out into different directory trees? If it’s necessary to have separate dircetory trees per application, then either the transpiled code would need to be copied to each destination tree or there would need to be a URI for transpiled code and another for the application.

Anyway, see how you get on :)

Cheers
John

On 21/02/2016, 19:23, "Dimitri" <***@cargosoft.ru> wrote:

John, Fritz,

Yes exactly - I was talking about profiling QxCompiler itself. At that
moment I was experimenting with shared database and was wondering why
it doesn't bring any speedup. I thought profiling would help to
understand where QxCompiler spends most time; later it turned to be
more fundamental issue.

I've experimented with a bit with incremental builds in our CI. So far,
I think incremental builds are fine for CI builds (triggered by
commits). That build directory isn't cleared each time didn't cause any
complications yet. It's not a problem that first build takes
significant time, it's only time of subsequent builds that matters.
It's also a good practice to perform clean builds for releases, but
slow release builds are quite acceptable since they don't occur
frequently.

However, I don't think efficient shared cache is impossible "by
design". I've looked into {qxt,myproject}/{source,build}-
output/transpiled/qx/*, did a byte-by-byte comparison and found that
*.js files are identical (therefore, they could probably be cached
across targets and libraries). Of course, there was difference in which
files were present, due to different dependencies.

(Besides, all the *.map files contain "null", are they really needed?)

Speaking of dependencies, I did some comparison between qxt and our
project:

Framework classes used: 283 vs. 378
Contribs: UploadMgr (both)
Uncompressed script: 4.8M vs. 6.4M
Compressed script (QxCompiler): 1.4M vs. 1.9M
Compressed script (generate.py): 700K vs. 1M

Definitely there is some room for optimization :)

John, sorry for delay in conversation. I'll be answering you on server-
side JS soon.

Dimitri
John Spackman
2016-02-19 07:56:05 UTC
Permalink
Hi Dimitri

You’ve done everything right but I’ve realised my mistake is that it’s not just the database any more, it’s the transpiled code is specific to each target and that it what’s taking the time. So the correct solution is that I must change QxCompiler to that several applications can be built within one target directory.

This won’t be a probem, but I don’t think I will have time to do it today; I should be able to sort it out this weekend though

BTW, all the classes have to be transpiled for every build; this is because although we know that in reality, Qooxdoo and all the contribs are all ES5 that could change so we have to assume they’re all ES6. Plus there is code removal (eg removing qx.debug variants for build target) and we inject dependency information for use during the boot loader (although that could be refactored)

John
John,
I gave it a shot, but not sure if I did it right. In create-app-
demo.js, I adjusted dbFilename to point to absolute location in /tmp.
First time compilation took ~30 seconds. Then I removed source-output,
as if it were a clean checkout, tried to recompile and observed no
speed-up at all.
It suddenly came to my mind: does Babel transpile all the (required)
qooxdoo classes each time? I think it's a redundant step since qooxdoo
itself is pretty much ES5 compliant.
I'm not a Node.js pro (I'm a JavaEE architect in fact :) so I'm not
experienced in profiling Node applications. It would be nice if some
verbose output (with timings) was produced by QxCompiler, as a poor
man's profiling facility :)
Dimitri
P.S. Sorry for "practise" instead of "practice", I usually don't do
such stupid mistakes. I even looked it up in the dictionary before
writing a message, but finally screwed everything up :-D
No problem; the next release will have a dbFilename property on
// Makers use an Analyser to figure out what the Target should write
var maker = new qxcompiler.makers.SimpleApp("qxt.Application",
"qxt.theme.Theme").set({
// Targets know how to output an application
target: new
qxcompiler.targets.SourceTarget("../testdata/qxt/source-output"),
locales: [ "en", "es" ],
dbFilename: "my-qxcompiler-db.json"
});
dbFilename is relative to the target output dir, or it can be
absolute.
John
Feature request: shared cache
=============================
It's common practise in continuous integration that CI server does a
clean checkout per each build. Incremental builds are common practise
too, but they can be unreliable and problematic, especially for
complex
builds.
At the moment, QxCompiler maintains a per-project cache that is
populated on first compilation. This might take significant time (up
to
1 minute; subsequent builds are incredibly fast though). Original
qooxdoo generator solves this by using platform-wide cache, usually
in
$TMPDIR/qx{version}/cache. This also speeds up compilation if there
are
several projects using the same qooxdoo version.
It would be nice if QxCompiler supported shared cache in the same
manner. Please mind that configurable cache path is important.
Hardcoded $TMPDIR/smth works bad, because on modern Linux distros
$TMPDIR lives in ramdisk and is cleared at each boot.
Dimitri
Hi all
There is a first release of my QxCompiler that adds ES6 to Qooxdoo
applications and replaces the generate.py toolchain with a faster,
100% Javascript tool that is easily extensible.
You can find the first release at GitHub here: https://github.com/joh
nspackman/qxcompiler
It’s an alpha release, but something that’s been in development for a
while now and which I’m starting to build into my production
servers.
I’m very open to pull requests or collaboration, and keen to see this
become a useful tool for myself as well as others so any questions
etc please ask.
Regards
John
-------------------------------------------------------------------
-----------
Site24x7 APM Insight: Get Deep Visibility into Application
Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
-------------------------------------------------------------------
-----------
Site24x7 APM Insight: Get Deep Visibility into Application
Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
-------------------------------------------------------------------
-----------
Site24x7 APM Insight: Get Deep Visibility into Application
Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
Dimitri
2016-02-22 16:58:29 UTC
Permalink
Feature request: background recompiler + web server + reloader
In qooxdoo development, it's common practice to use a micro HTTP server
to run applications, eg. python -m SimpleHTTPServer, generate.py
source-server, editor/IDE embedded etc. This is because of limitations
of file:/// addressing scheme and CORS/XHR issues that it has.
QxCompiler is going to introduce another background process for live
code transpiling. Why not combining these two, so that the developer
doesn't have to launch two separate processes?
It should be pretty easy to serve static contents from Node. In the
future, automatic application reloading could be implemented, something
that is available now with generate.py watch + source-server-reload
jobs.
Dimitri
Post by John Spackman
Hi all
There is a first release of my QxCompiler that adds ES6 to Qooxdoo
applications and replaces the generate.py toolchain with a faster,
100% Javascript tool that is easily extensible.
https://github.com/johnspackman/qxcompiler
It’s an alpha release, but something that’s been in development for a
while now and which I’m starting to build into my production
servers.  
I’m very open to pull requests or collaboration, and keen to see this
become a useful tool for myself as well as others so any questions
etc please ask.
Regards
John
-------------------------------------------------------------------
-----------
Site24x7 APM Insight: Get Deep Visibility into Application
Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
Adrian Haarbach
2016-02-22 17:38:12 UTC
Permalink
Concerning the webserver, I am using Grunt and the grunt-http-server
<https://www.npmjs.com/package/grunt-http-server> plugin, which is based on
node.js and much faster than python's SimpleHttpServer in serving files.
Also, it allows to specify a proxy where it redirects all requests it
cannot resolve locally (e.g requests to tomcat). Really useful when
developing the web part of an application while the backend runs somewhere
remotely.

qooxdoo already comes with a package.json and Gruntfile.js by default now,
I just edited them as below to make the webserver work:


*package.json:*

{
"name": "project",
"version": "0.1.0",
"repository": {},
"devDependencies": {
"grunt": "~0.4.2",
"grunt-http-server": "^1.13.0"
}
"license": "Apache-2.0"
}



*Gruntfile.js:*

module.exports = function(grunt) {
grunt.initConfig({
'http-server': {

'dev': {

// the server root directory
root: ".",

// the server port, can also be written as a function, e.g.
// port: function() { return 8282; }
port: 9999,

// the host ip address
// If specified to, for example, "127.0.0.1" the server will
only be available on that ip.
// Specify "0.0.0.0" to be available everywhere
host: "127.0.0.1",

// server default file extension
ext: "html",

// run in parallel with other tasks
runInBackground: false,

// Proxies all requests which can't be resolved locally to
the given url
// Note this this will disable 'showDir'
proxy: "http://mybackendserver.com",
}
}
});
// // 3. Where we tell Grunt we plan to use this plug-in.
grunt.loadNpmTasks('grunt-http-server');
// // 4. Where we tell Grunt what to do when we type "grunt" into the
terminal.
grunt.registerTask('default', ['http-server:dev']);
};



*Then to install all dependencies and run, just do:*

npm install -g grunt-cli
npm install //installs dependencies listed in package.json into
node_modules folder
grunt //runs default http-server:dev job



The recompiler and reloader for dev jobs and additionally concatenator and
uglifiers/minifiers for dist jobs can also be added as simple grunt tasks.
There are lots of plugins for these tasks.

Looking forward to ditch ./generate.py in favor of a simple and faster
grunt job!

Adrian
Post by Dimitri
Feature request: background recompiler + web server + reloader
In qooxdoo development, it's common practice to use a micro HTTP server to
run applications, eg. python -m SimpleHTTPServer, generate.py
source-server, editor/IDE embedded etc. This is because of limitations of
file:/// addressing scheme and CORS/XHR issues that it has.
QxCompiler is going to introduce another background process for live code
transpiling. Why not combining these two, so that the developer doesn't
have to launch two separate processes?
It should be pretty easy to serve static contents from Node. In the
future, automatic application reloading could be implemented, something
that is available now with generate.py watch + source-server-reload jobs.
Dimitri
Hi all
There is a first release of my QxCompiler that adds ES6 to Qooxdoo
applications and replaces the generate.py toolchain with a faster, 100%
Javascript tool that is easily extensible.
https://github.com/johnspackman/qxcompiler
It’s an alpha release, but something that’s been in development for a
while now and which I’m starting to build into my production servers.
I’m very open to pull requests or collaboration, and keen to see this
become a useful tool for myself as well as others so any questions etc
please ask.
Regards
John
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
John Spackman
2016-02-22 18:57:38 UTC
Permalink
Hi Adrian

Then to install all dependencies and run, just do:
npm install -g grunt-cli
npm install //installs dependencies listed in package.json into node_modules folder
grunt //runs default http-server:dev job

Looking forward to ditch ./generate.py in favor of a simple and faster grunt job!

That sounds really cool - I don’t have any experience with grunt but that’s definitely the right progression; any chance you could write a grunt job that would allow exactly that kind of usage, and if so what would you need? I imagine that you wouldn’t need a command line version of QxCompiler, so long as there is an npm installable package?

Cheers
John
Dimitri
2016-02-22 19:03:25 UTC
Permalink
Adrian,

Sounds pretty cool! Unfortunately there's one missing link here -
QxCompiler is not a Grunt task yet.

BTW, is Grunt capable of running two or more background tasks
simultaneously? eg. QxCompiler job + web server
Or do we need to launch them in separate Grunt instances?

Dimitri
Post by Adrian Haarbach
Concerning the webserver, I am using Grunt and the grunt-http-server
plugin, which is based on node.js and much faster than python's
SimpleHttpServer in serving files.
Also, it allows to specify a proxy where it redirects all requests it
cannot resolve locally (e.g requests to tomcat). Really useful when
developing the web part of an application while the backend runs
somewhere remotely.
qooxdoo already comes with a package.json and Gruntfile.js by default
{
  "name": "project",
  "version": "0.1.0",
  "repository": {},
  "devDependencies": {
    "grunt": "~0.4.2",
    "grunt-http-server": "^1.13.0"
  }
  "license": "Apache-2.0"
}
module.exports = function(grunt) {
  grunt.initConfig({   
      'http-server': {
   
          'dev': {
   
              // the server root directory 
              root: ".",
   
              // the server port, can also be written as a function,
e.g. 
              // port: function() { return 8282; } 
              port: 9999,
   
              // the host ip address 
              // If specified to, for example, "127.0.0.1" the server
will only be available on that ip. 
              // Specify "0.0.0.0" to be available everywhere 
              host: "127.0.0.1",
   
              // server default file extension 
              ext: "html",
   
              // run in parallel with other tasks 
              runInBackground: false,
   
              // Proxies all requests which can't be resolved locally
to the given url 
              // Note this this will disable 'showDir' 
              proxy: "http://mybackendserver.com",
          } 
      }
  });
  // // 3. Where we tell Grunt we plan to use this plug-in.
  grunt.loadNpmTasks('grunt-http-server');
  // // 4. Where we tell Grunt what to do when we type "grunt" into
the terminal.
  grunt.registerTask('default', ['http-server:dev']);
};
npm install -g grunt-cli
npm install   //installs dependencies listed in package.json into
node_modules folder
grunt //runs default http-server:dev job
The recompiler and reloader for dev jobs and additionally
concatenator and uglifiers/minifiers for dist jobs can also be added
as simple grunt tasks. There are lots of plugins for these tasks.
Looking forward to ditch ./generate.py in favor of a simple and
faster grunt job!
Adrian
Post by Dimitri
Feature request: background recompiler + web server + reloader
In qooxdoo development, it's common practice to use a micro HTTP
server to run applications, eg. python -m SimpleHTTPServer,
generate.py source-server, editor/IDE embedded etc. This is because
of limitations of file:/// addressing scheme and CORS/XHR issues
that it has.
QxCompiler is going to introduce another background process for
live code transpiling. Why not combining these two, so that the
developer doesn't have to launch two separate processes?
It should be pretty easy to serve static contents from Node. In the
future, automatic application reloading could be implemented,
something that is available now with generate.py watch + source-
server-reload jobs.
Dimitri
Post by John Spackman
Hi all
There is a first release of my QxCompiler that adds ES6 to
Qooxdoo applications and replaces the generate.py toolchain with
a faster, 100% Javascript tool that is easily extensible.
You can find the first release at GitHub here: https://github.com
/johnspackman/qxcompiler
It’s an alpha release, but something that’s been in development
for a while now and which I’m starting to build into my
production servers.  
I’m very open to pull requests or collaboration, and keen to see
this become a useful tool for myself as well as others so any
questions etc please ask.
Regards
John
---------------------------------------------------------------
---------------
Site24x7 APM Insight: Get Deep Visibility into Application
Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
-----------------------------------------------------------------
-------------
Site24x7 APM Insight: Get Deep Visibility into Application
Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
-------------------------------------------------------------------
-----------
Site24x7 APM Insight: Get Deep Visibility into Application
Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
thron7
2016-02-22 21:02:34 UTC
Permalink
Post by Adrian Haarbach
*Then to install all dependencies and run, just do:*
npm install -g grunt-cli
npm install //installs dependencies listed in package.json into
node_modules folder
grunt //runs default http-server:dev job
The recompiler and reloader for dev jobs and additionally concatenator and
uglifiers/minifiers for dist jobs can also be added as simple grunt tasks.
There are lots of plugins for these tasks.
Looking forward to ditch ./generate.py in favor of a simple and faster
grunt job!
Just as a quick reminder, as I'm not sure everybody is aware of this:

Until not so long ago the qooxdoo repo had a fair amount of code to utilize
Grunt as the new build tool, aiming to replace the Generator. This support
has then be cut back to the utmost minimum somewhen last year, but the full
code is still available in the 'next' repo,

https://github.com/qooxdoo/next/tree/master/tool/grunt

It already implemented the core Generator jobs in JS, making use of the
Node ecosystem of libraries. IIRC this included 'source' and 'build' jobs,
together with dependency analysis, resource handling and loader generation.
Jobs not already implemented in JS are transparently passed through to the
old Generator, allowing for a stepwise replacement of the Python
implementation while still providing the full functionality all the time.

I could see Grunt being a command-line frontend for the jobs, with
QxCompiler remaining a pure API-based library which is included as a
dependency and utilized from Grunt files.

Even if you feel the code in qooxdoo.next is too 'Grunt-heavy', or would
just like to hack on an all-new, own version of Grunt support (which I can
always understand ;-)), the code in qooxdoo.next might be a good
inspiration to draw from.

T.
John Spackman
2016-02-22 19:34:01 UTC
Permalink
This should be straightforward once the continuous compiler process is done – I will implement that as an API too, so integrating with an HTTP server implementation would be trivial and could be done as a separate contrib or PR. I’ve been using https://www.npmjs.com/package/http-server and find it fast and functional; a brief look at the code shows that it’s pretty lightweight

John

From: Dimitri <***@cargosoft.ru>
Reply-To: qooxdoo Development <qooxdoo-***@lists.sourceforge.net>
Date: Monday, 22 February 2016 at 16:58
To: <qooxdoo-***@lists.sourceforge.net>
Subject: Re: [qooxdoo-devel] QxCompiler - add ES6, faster compilation, and 100% Javascript API to building applications

Feature request: background recompiler + web server + reloader

In qooxdoo development, it's common practice to use a micro HTTP server to run applications, eg. python -m SimpleHTTPServer, generate.py source-server, editor/IDE embedded etc. This is because of limitations of file:/// addressing scheme and CORS/XHR issues that it has.
QxCompiler is going to introduce another background process for live code transpiling. Why not combining these two, so that the developer doesn't have to launch two separate processes?
It should be pretty easy to serve static contents from Node. In the future, automatic application reloading could be implemented, something that is available now with generate.py watch + source-server-reload jobs.

Dimitri
d***@cost-savers.net
2016-02-22 17:44:51 UTC
Permalink
Adrian,

This is great. We have thought of using it instead of simple....
Now you have done the job. Is it possible to make it universal and into pull requestable files...?
We will put it on our todo list to exchange the simple python server...

Stefan
Post by Adrian Haarbach
Concerning the webserver, I am using Grunt and the grunt-http-server
<https://www.npmjs.com/package/grunt-http-server> plugin, which is based on
node.js and much faster than python's SimpleHttpServer in serving files.
Also, it allows to specify a proxy where it redirects all requests it
cannot resolve locally (e.g requests to tomcat). Really useful when
developing the web part of an application while the backend runs somewhere
remotely.
qooxdoo already comes with a package.json and Gruntfile.js by default now,
*package.json:*
{
"name": "project",
"version": "0.1.0",
"repository": {},
"devDependencies": {
"grunt": "~0.4.2",
"grunt-http-server": "^1.13.0"
}
"license": "Apache-2.0"
}
*Gruntfile.js:*
module.exports = function(grunt) {
grunt.initConfig({
'http-server': {
'dev': {
// the server root directory
root: ".",
// the server port, can also be written as a function, e.g.
// port: function() { return 8282; }
port: 9999,
// the host ip address
// If specified to, for example, "127.0.0.1" the server will
only be available on that ip.
// Specify "0.0.0.0" to be available everywhere
host: "127.0.0.1",
// server default file extension
ext: "html",
// run in parallel with other tasks
runInBackground: false,
// Proxies all requests which can't be resolved locally to
the given url
// Note this this will disable 'showDir'
proxy: "http://mybackendserver.com",
}
}
});
// // 3. Where we tell Grunt we plan to use this plug-in.
grunt.loadNpmTasks('grunt-http-server');
// // 4. Where we tell Grunt what to do when we type "grunt" into the
terminal.
grunt.registerTask('default', ['http-server:dev']);
};
*Then to install all dependencies and run, just do:*
npm install -g grunt-cli
npm install //installs dependencies listed in package.json into
node_modules folder
grunt //runs default http-server:dev job
The recompiler and reloader for dev jobs and additionally concatenator and
uglifiers/minifiers for dist jobs can also be added as simple grunt tasks.
There are lots of plugins for these tasks.
Looking forward to ditch ./generate.py in favor of a simple and faster
grunt job!
Adrian
Post by Dimitri
Feature request: background recompiler + web server + reloader
In qooxdoo development, it's common practice to use a micro HTTP server to
run applications, eg. python -m SimpleHTTPServer, generate.py
source-server, editor/IDE embedded etc. This is because of limitations of
file:/// addressing scheme and CORS/XHR issues that it has.
QxCompiler is going to introduce another background process for live code
transpiling. Why not combining these two, so that the developer doesn't
have to launch two separate processes?
It should be pretty easy to serve static contents from Node. In the
future, automatic application reloading could be implemented, something
that is available now with generate.py watch + source-server-reload jobs.
Dimitri
Hi all
There is a first release of my QxCompiler that adds ES6 to Qooxdoo
applications and replaces the generate.py toolchain with a faster, 100%
Javascript tool that is easily extensible.
https://github.com/johnspackman/qxcompiler
It’s an alpha release, but something that’s been in development for a
while now and which I’m starting to build into my production servers.
I’m very open to pull requests or collaboration, and keen to see this
become a useful tool for myself as well as others so any questions etc
please ask.
Regards
John
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
Adrian Haarbach
2016-02-22 18:46:31 UTC
Permalink
Hi Stefan,

here is the pull request: https://github.com/qooxdoo/qooxdoo/pull/175:

Added grunt task source-server-nodejs to the grunt templates.
Now you have the choice between
grunt source-server -> python SimpleHttpServer
grunt source-server-nodejs -> node grunt-http-server

cheers,
Adrian
Post by d***@cost-savers.net
Adrian,
This is great. We have thought of using it instead of simple....
Now you have done the job. Is it possible to make it universal and into
pull requestable files...?
We will put it on our todo list to exchange the simple python server...
Stefan
Post by Adrian Haarbach
Concerning the webserver, I am using Grunt and the grunt-http-server
<https://www.npmjs.com/package/grunt-http-server> plugin, which is
based on
Post by Adrian Haarbach
node.js and much faster than python's SimpleHttpServer in serving files.
Also, it allows to specify a proxy where it redirects all requests it
cannot resolve locally (e.g requests to tomcat). Really useful when
developing the web part of an application while the backend runs
somewhere
Post by Adrian Haarbach
remotely.
qooxdoo already comes with a package.json and Gruntfile.js by default
now,
Post by Adrian Haarbach
*package.json:*
{
"name": "project",
"version": "0.1.0",
"repository": {},
"devDependencies": {
"grunt": "~0.4.2",
"grunt-http-server": "^1.13.0"
}
"license": "Apache-2.0"
}
*Gruntfile.js:*
module.exports = function(grunt) {
grunt.initConfig({
'http-server': {
'dev': {
// the server root directory
root: ".",
// the server port, can also be written as a function, e.g.
// port: function() { return 8282; }
port: 9999,
// the host ip address
// If specified to, for example, "127.0.0.1" the server
will
Post by Adrian Haarbach
only be available on that ip.
// Specify "0.0.0.0" to be available everywhere
host: "127.0.0.1",
// server default file extension
ext: "html",
// run in parallel with other tasks
runInBackground: false,
// Proxies all requests which can't be resolved locally to
the given url
// Note this this will disable 'showDir'
proxy: "http://mybackendserver.com",
}
}
});
// // 3. Where we tell Grunt we plan to use this plug-in.
grunt.loadNpmTasks('grunt-http-server');
// // 4. Where we tell Grunt what to do when we type "grunt" into the
terminal.
grunt.registerTask('default', ['http-server:dev']);
};
*Then to install all dependencies and run, just do:*
npm install -g grunt-cli
npm install //installs dependencies listed in package.json into
node_modules folder
grunt //runs default http-server:dev job
The recompiler and reloader for dev jobs and additionally concatenator
and
Post by Adrian Haarbach
uglifiers/minifiers for dist jobs can also be added as simple grunt
tasks.
Post by Adrian Haarbach
There are lots of plugins for these tasks.
Looking forward to ditch ./generate.py in favor of a simple and faster
grunt job!
Adrian
Post by Dimitri
Feature request: background recompiler + web server + reloader
In qooxdoo development, it's common practice to use a micro HTTP server
to
Post by Adrian Haarbach
Post by Dimitri
run applications, eg. python -m SimpleHTTPServer, generate.py
source-server, editor/IDE embedded etc. This is because of limitations
of
Post by Adrian Haarbach
Post by Dimitri
file:/// addressing scheme and CORS/XHR issues that it has.
QxCompiler is going to introduce another background process for live
code
Post by Adrian Haarbach
Post by Dimitri
transpiling. Why not combining these two, so that the developer doesn't
have to launch two separate processes?
It should be pretty easy to serve static contents from Node. In the
future, automatic application reloading could be implemented, something
that is available now with generate.py watch + source-server-reload
jobs.
Post by Adrian Haarbach
Post by Dimitri
Dimitri
Hi all
There is a first release of my QxCompiler that adds ES6 to Qooxdoo
applications and replaces the generate.py toolchain with a faster, 100%
Javascript tool that is easily extensible.
https://github.com/johnspackman/qxcompiler
It’s an alpha release, but something that’s been in development for a
while now and which I’m starting to build into my production servers.
I’m very open to pull requests or collaboration, and keen to see this
become a useful tool for myself as well as others so any questions etc
please ask.
Regards
John
------------------------------------------------------------------------------
Post by Adrian Haarbach
Post by Dimitri
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
------------------------------------------------------------------------------
Post by Adrian Haarbach
Post by Dimitri
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
------------------------------------------------------------------------------
Post by Adrian Haarbach
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
d***@cost-savers.net
2016-02-22 19:14:07 UTC
Permalink
Adrian,

Fantastic!

This shows how strong and fast a community can be.

Thanks!

Stefan
Post by Adrian Haarbach
Hi Stefan,
Added grunt task source-server-nodejs to the grunt templates.
Now you have the choice between
grunt source-server -> python SimpleHttpServer
grunt source-server-nodejs -> node grunt-http-server
cheers,
Adrian
Post by d***@cost-savers.net
Adrian,
This is great. We have thought of using it instead of simple....
Now you have done the job. Is it possible to make it universal and into
pull requestable files...?
We will put it on our todo list to exchange the simple python server...
Stefan
Post by Adrian Haarbach
Concerning the webserver, I am using Grunt and the grunt-http-server
<https://www.npmjs.com/package/grunt-http-server> plugin, which is
based on
Post by Adrian Haarbach
node.js and much faster than python's SimpleHttpServer in serving files.
Also, it allows to specify a proxy where it redirects all requests it
cannot resolve locally (e.g requests to tomcat). Really useful when
developing the web part of an application while the backend runs
somewhere
Post by Adrian Haarbach
remotely.
qooxdoo already comes with a package.json and Gruntfile.js by default
now,
Post by Adrian Haarbach
*package.json:*
{
"name": "project",
"version": "0.1.0",
"repository": {},
"devDependencies": {
"grunt": "~0.4.2",
"grunt-http-server": "^1.13.0"
}
"license": "Apache-2.0"
}
*Gruntfile.js:*
module.exports = function(grunt) {
grunt.initConfig({
'http-server': {
'dev': {
// the server root directory
root: ".",
// the server port, can also be written as a function, e.g.
// port: function() { return 8282; }
port: 9999,
// the host ip address
// If specified to, for example, "127.0.0.1" the server
will
Post by Adrian Haarbach
only be available on that ip.
// Specify "0.0.0.0" to be available everywhere
host: "127.0.0.1",
// server default file extension
ext: "html",
// run in parallel with other tasks
runInBackground: false,
// Proxies all requests which can't be resolved locally to
the given url
// Note this this will disable 'showDir'
proxy: "http://mybackendserver.com",
}
}
});
// // 3. Where we tell Grunt we plan to use this plug-in.
grunt.loadNpmTasks('grunt-http-server');
// // 4. Where we tell Grunt what to do when we type "grunt" into the
terminal.
grunt.registerTask('default', ['http-server:dev']);
};
*Then to install all dependencies and run, just do:*
npm install -g grunt-cli
npm install //installs dependencies listed in package.json into
node_modules folder
grunt //runs default http-server:dev job
The recompiler and reloader for dev jobs and additionally concatenator
and
Post by Adrian Haarbach
uglifiers/minifiers for dist jobs can also be added as simple grunt
tasks.
Post by Adrian Haarbach
There are lots of plugins for these tasks.
Looking forward to ditch ./generate.py in favor of a simple and faster
grunt job!
Adrian
Post by Dimitri
Feature request: background recompiler + web server + reloader
In qooxdoo development, it's common practice to use a micro HTTP server
to
Post by Adrian Haarbach
Post by Dimitri
run applications, eg. python -m SimpleHTTPServer, generate.py
source-server, editor/IDE embedded etc. This is because of limitations
of
Post by Adrian Haarbach
Post by Dimitri
file:/// addressing scheme and CORS/XHR issues that it has.
QxCompiler is going to introduce another background process for live
code
Post by Adrian Haarbach
Post by Dimitri
transpiling. Why not combining these two, so that the developer doesn't
have to launch two separate processes?
It should be pretty easy to serve static contents from Node. In the
future, automatic application reloading could be implemented, something
that is available now with generate.py watch + source-server-reload
jobs.
Post by Adrian Haarbach
Post by Dimitri
Dimitri
Hi all
There is a first release of my QxCompiler that adds ES6 to Qooxdoo
applications and replaces the generate.py toolchain with a faster, 100%
Javascript tool that is easily extensible.
https://github.com/johnspackman/qxcompiler
It’s an alpha release, but something that’s been in development for a
while now and which I’m starting to build into my production servers.
I’m very open to pull requests or collaboration, and keen to see this
become a useful tool for myself as well as others so any questions etc
please ask.
Regards
John
------------------------------------------------------------------------------
Post by Adrian Haarbach
Post by Dimitri
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
------------------------------------------------------------------------------
Post by Adrian Haarbach
Post by Dimitri
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
------------------------------------------------------------------------------
Post by Adrian Haarbach
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
d***@cost-savers.net
2016-02-22 19:18:59 UTC
Permalink
Dimitri,

Independently, of course...

Stefan
Post by d***@cost-savers.net
Adrian,
Sounds pretty cool! Unfortunately there's one missing link here -
QxCompiler is not a Grunt task yet.
BTW, is Grunt capable of running two or more background tasks
simultaneously? eg. QxCompiler job + web server
Or do we need to launch them in separate Grunt instances?
Dimitri
Post by Adrian Haarbach
Concerning the webserver, I am using Grunt and the grunt-http-server
plugin, which is based on node.js and much faster than python's
SimpleHttpServer in serving files.
Also, it allows to specify a proxy where it redirects all requests it
cannot resolve locally (e.g requests to tomcat). Really useful when
developing the web part of an application while the backend runs
somewhere remotely.
qooxdoo already comes with a package.json and Gruntfile.js by default
{
  "name": "project",
  "version": "0.1.0",
  "repository": {},
  "devDependencies": {
    "grunt": "~0.4.2",
    "grunt-http-server": "^1.13.0"
  }
  "license": "Apache-2.0"
}
module.exports = function(grunt) {
  grunt.initConfig({   
      'http-server': {
   
          'dev': {
   
              // the server root directory 
              root: ".",
   
              // the server port, can also be written as a function,
e.g. 
              // port: function() { return 8282; } 
              port: 9999,
   
              // the host ip address 
              // If specified to, for example, "127.0.0.1" the server
will only be available on that ip. 
              // Specify "0.0.0.0" to be available everywhere 
              host: "127.0.0.1",
   
              // server default file extension 
              ext: "html",
   
              // run in parallel with other tasks 
              runInBackground: false,
   
              // Proxies all requests which can't be resolved locally
to the given url 
              // Note this this will disable 'showDir' 
              proxy: "http://mybackendserver.com",
          } 
      }
  });
  // // 3. Where we tell Grunt we plan to use this plug-in.
  grunt.loadNpmTasks('grunt-http-server');
  // // 4. Where we tell Grunt what to do when we type "grunt" into
the terminal.
  grunt.registerTask('default', ['http-server:dev']);
};
npm install -g grunt-cli
npm install   //installs dependencies listed in package.json into
node_modules folder
grunt //runs default http-server:dev job
The recompiler and reloader for dev jobs and additionally
concatenator and uglifiers/minifiers for dist jobs can also be added
as simple grunt tasks. There are lots of plugins for these tasks.
Looking forward to ditch ./generate.py in favor of a simple and
faster grunt job!
Adrian
Post by Dimitri
Feature request: background recompiler + web server + reloader
In qooxdoo development, it's common practice to use a micro HTTP
server to run applications, eg. python -m SimpleHTTPServer,
generate.py source-server, editor/IDE embedded etc. This is because
of limitations of file:/// addressing scheme and CORS/XHR issues
that it has.
QxCompiler is going to introduce another background process for
live code transpiling. Why not combining these two, so that the
developer doesn't have to launch two separate processes?
It should be pretty easy to serve static contents from Node. In the
future, automatic application reloading could be implemented,
something that is available now with generate.py watch + source-
server-reload jobs.
Dimitri
Post by John Spackman
Hi all
There is a first release of my QxCompiler that adds ES6 to
Qooxdoo applications and replaces the generate.py toolchain with
a faster, 100% Javascript tool that is easily extensible.
You can find the first release at GitHub here: https://github.com
/johnspackman/qxcompiler
It’s an alpha release, but something that’s been in development
for a while now and which I’m starting to build into my
production servers.  
I’m very open to pull requests or collaboration, and keen to see
this become a useful tool for myself as well as others so any
questions etc please ask.
Regards
John
---------------------------------------------------------------
---------------
Site24x7 APM Insight: Get Deep Visibility into Application
Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions
now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
-----------------------------------------------------------------
-------------
Site24x7 APM Insight: Get Deep Visibility into Application
Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
-------------------------------------------------------------------
-----------
Site24x7 APM Insight: Get Deep Visibility into Application
Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
Dimitri
2016-02-23 01:02:45 UTC
Permalink
Stefan, Adrian,

Quick googling shows that there are several grunt plugins around that
allow to execute tasks simultaneously: grunt-parallel, grunt-
parallelize, grunt-concurrent. The latter looks maintained and
promising in all the ways. As a proof of concept, could someone try to
run, say, two HTTP servers under Grunt? You're guys are much more
experienced in Grunt than me, so it will take significantly less time
:) If it works, that would mean we already have a mechanism to run
QxCompiler's continuous recompiler job with HTTP server in a single
Grunt instance.

Dimitri
Post by d***@cost-savers.net
Dimitri,
Independently, of course...
Stefan
Post by d***@cost-savers.net
Adrian,
Sounds pretty cool! Unfortunately there's one missing link here -
QxCompiler is not a Grunt task yet.
BTW, is Grunt capable of running two or more background tasks
simultaneously? eg. QxCompiler job + web server
Or do we need to launch them in separate Grunt instances?
Dimitri
Post by Adrian Haarbach
Concerning the webserver, I am using Grunt and the grunt-http-
server
plugin, which is based on node.js and much faster than python's
SimpleHttpServer in serving files.
Also, it allows to specify a proxy where it redirects all
requests it
cannot resolve locally (e.g requests to tomcat). Really useful when
developing the web part of an application while the backend runs
somewhere remotely.
qooxdoo already comes with a package.json and Gruntfile.js by default
{
  "name": "project",
  "version": "0.1.0",
  "repository": {},
  "devDependencies": {
    "grunt": "~0.4.2",
    "grunt-http-server": "^1.13.0"
  }
  "license": "Apache-2.0"
}
module.exports = function(grunt) {
  grunt.initConfig({   
      'http-server': {
   
          'dev': {
   
              // the server root directory 
              root: ".",
   
              // the server port, can also be written as a function,
e.g. 
              // port: function() { return 8282; } 
              port: 9999,
   
              // the host ip address 
              // If specified to, for example, "127.0.0.1" the server
will only be available on that ip. 
              // Specify "0.0.0.0" to be available everywhere 
              host: "127.0.0.1",
   
              // server default file extension 
              ext: "html",
   
              // run in parallel with other tasks 
              runInBackground: false,
   
              // Proxies all requests which can't be resolved locally
to the given url 
              // Note this this will disable 'showDir' 
              proxy: "http://mybackendserver.com",
          } 
      }
  });
  // // 3. Where we tell Grunt we plan to use this plug-in.
  grunt.loadNpmTasks('grunt-http-server');
  // // 4. Where we tell Grunt what to do when we type "grunt" into
the terminal.
  grunt.registerTask('default', ['http-server:dev']);
};
npm install -g grunt-cli
npm install   //installs dependencies listed in package.json into
node_modules folder
grunt //runs default http-server:dev job
The recompiler and reloader for dev jobs and additionally
concatenator and uglifiers/minifiers for dist jobs can also be added
as simple grunt tasks. There are lots of plugins for these tasks.
Looking forward to ditch ./generate.py in favor of a simple and
faster grunt job!
Adrian
Post by Dimitri
Feature request: background recompiler + web server + reloader
In qooxdoo development, it's common practice to use a micro HTTP
server to run applications, eg. python -m SimpleHTTPServer,
generate.py source-server, editor/IDE embedded etc. This is because
of limitations of file:/// addressing scheme and CORS/XHR issues
that it has.
QxCompiler is going to introduce another background process for
live code transpiling. Why not combining these two, so that the
developer doesn't have to launch two separate processes?
It should be pretty easy to serve static contents from Node. In the
future, automatic application reloading could be implemented,
something that is available now with generate.py watch +
source-
server-reload jobs.
Dimitri
Post by John Spackman
Hi all
There is a first release of my QxCompiler that adds ES6 to
Qooxdoo applications and replaces the generate.py toolchain with
a faster, 100% Javascript tool that is easily extensible.
You can find the first release at GitHub here: https://github
.com
/johnspackman/qxcompiler
It’s an alpha release, but something that’s been in development
for a while now and which I’m starting to build into my
production servers.  
I’m very open to pull requests or collaboration, and keen to see
this become a useful tool for myself as well as others so any
questions etc please ask.
Regards
John
-----------------------------------------------------------
----
---------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions
now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4
140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
-------------------------------------------------------------
----
-------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just
$35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/414
0
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
---------------------------------------------------------------
----
-----------
Site24x7 APM Insight: Get Deep Visibility into Application
Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
-----------------------------------------------------------------
-------------
Site24x7 APM Insight: Get Deep Visibility into Application
Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
-------------------------------------------------------------------
-----------
Site24x7 APM Insight: Get Deep Visibility into Application
Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
d***@cost-savers.net
2016-02-22 19:26:17 UTC
Permalink
John,

We really like your approach of the qxcompiler.
I think it is very important to keep it clean and clear-cut as a tool and not complicate it now. There might be many wishes to do this and that. It is good if not moved too fast before knowing what is best for the subproject.

Your ideas behind is really promising. You have done a fantastic job!

We do believe it is time to retire Generate.py and therefore we would like to see qxcompiler taking over all the job of Generate.py.

Speed, performance, simplicity and added functionality is so far better than Generate.py...(what has been implemented so far)
We are using it in a smaller subproject of an admin desktop application now and it is getting better and better. We had to solve some issues.

We will soon come back with more feedback when we see it is grounded...

Stefan
Post by John Spackman
Hi Adrian
npm install -g grunt-cli
npm install //installs dependencies listed in package.json into node_modules folder
grunt //runs default http-server:dev job
Looking forward to ditch ./generate.py in favor of a simple and faster grunt job!
That sounds really cool - I don’t have any experience with grunt but that’s definitely the right progression; any chance you could write a grunt job that would allow exactly that kind of usage, and if so what would you need? I imagine that you wouldn’t need a command line version of QxCompiler, so long as there is an npm installable package?
Cheers
John
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
John Spackman
2016-02-22 19:47:37 UTC
Permalink
Hi Stefan

Thanks for the support, Im glad you like it :)

I agree that QxCompiler needs to stay focused on compiling, IMHO generate.py became the swiss army knife of the Qooxdoo toolchain and being written in Python and not as an API made made it opaque (for my purposes, at least)

Generate.py can be replaced for the general case of building apps “real soon now”, I think it needs a command line and continuous compilation and it’s ready (plus testing of course ;) )

In the wider case, there are some features that QxCompiler does not replace, EG API docs and test runner - although I havn’t looked at these yet, both should be fairly straightforward to do but also can be put off for now.

What issues have you had to solve? I’m open to pull requests ;)

Cheers
John
Post by d***@cost-savers.net
John,
We really like your approach of the qxcompiler.
I think it is very important to keep it clean and clear-cut as a tool and not complicate it now. There might be many wishes to do this and that. It is good if not moved too fast before knowing what is best for the subproject.
Your ideas behind is really promising. You have done a fantastic job!
We do believe it is time to retire Generate.py and therefore we would like to see qxcompiler taking over all the job of Generate.py.
Speed, performance, simplicity and added functionality is so far better than Generate.py...(what has been implemented so far)
We are using it in a smaller subproject of an admin desktop application now and it is getting better and better. We had to solve some issues.
We will soon come back with more feedback when we see it is grounded...
Stefan
Post by John Spackman
Hi Adrian
npm install -g grunt-cli
npm install //installs dependencies listed in package.json into node_modules folder
grunt //runs default http-server:dev job
Looking forward to ditch ./generate.py in favor of a simple and faster grunt job!
That sounds really cool - I don’t have any experience with grunt but that’s definitely the right progression; any chance you could write a grunt job that would allow exactly that kind of usage, and if so what would you need? I imagine that you wouldn’t need a command line version of QxCompiler, so long as there is an npm installable package?
Cheers
John
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
d***@cost-savers.net
2016-02-22 19:31:05 UTC
Permalink
Adrian,

Please make a doc pull request too... see the doc format etc to make it fit well into the manual...at the right place
If not the doc will lag...never good for new users!

Stefan
Post by Adrian Haarbach
Hi Stefan,
Added grunt task source-server-nodejs to the grunt templates.
Now you have the choice between
grunt source-server -> python SimpleHttpServer
grunt source-server-nodejs -> node grunt-http-server
cheers,
Adrian
Post by d***@cost-savers.net
Adrian,
This is great. We have thought of using it instead of simple....
Now you have done the job. Is it possible to make it universal and into
pull requestable files...?
We will put it on our todo list to exchange the simple python server...
Stefan
Post by Adrian Haarbach
Concerning the webserver, I am using Grunt and the grunt-http-server
<https://www.npmjs.com/package/grunt-http-server> plugin, which is
based on
Post by Adrian Haarbach
node.js and much faster than python's SimpleHttpServer in serving files.
Also, it allows to specify a proxy where it redirects all requests it
cannot resolve locally (e.g requests to tomcat). Really useful when
developing the web part of an application while the backend runs
somewhere
Post by Adrian Haarbach
remotely.
qooxdoo already comes with a package.json and Gruntfile.js by default
now,
Post by Adrian Haarbach
*package.json:*
{
"name": "project",
"version": "0.1.0",
"repository": {},
"devDependencies": {
"grunt": "~0.4.2",
"grunt-http-server": "^1.13.0"
}
"license": "Apache-2.0"
}
*Gruntfile.js:*
module.exports = function(grunt) {
grunt.initConfig({
'http-server': {
'dev': {
// the server root directory
root: ".",
// the server port, can also be written as a function, e.g.
// port: function() { return 8282; }
port: 9999,
// the host ip address
// If specified to, for example, "127.0.0.1" the server
will
Post by Adrian Haarbach
only be available on that ip.
// Specify "0.0.0.0" to be available everywhere
host: "127.0.0.1",
// server default file extension
ext: "html",
// run in parallel with other tasks
runInBackground: false,
// Proxies all requests which can't be resolved locally to
the given url
// Note this this will disable 'showDir'
proxy: "http://mybackendserver.com",
}
}
});
// // 3. Where we tell Grunt we plan to use this plug-in.
grunt.loadNpmTasks('grunt-http-server');
// // 4. Where we tell Grunt what to do when we type "grunt" into the
terminal.
grunt.registerTask('default', ['http-server:dev']);
};
*Then to install all dependencies and run, just do:*
npm install -g grunt-cli
npm install //installs dependencies listed in package.json into
node_modules folder
grunt //runs default http-server:dev job
The recompiler and reloader for dev jobs and additionally concatenator
and
Post by Adrian Haarbach
uglifiers/minifiers for dist jobs can also be added as simple grunt
tasks.
Post by Adrian Haarbach
There are lots of plugins for these tasks.
Looking forward to ditch ./generate.py in favor of a simple and faster
grunt job!
Adrian
Post by Dimitri
Feature request: background recompiler + web server + reloader
In qooxdoo development, it's common practice to use a micro HTTP server
to
Post by Adrian Haarbach
Post by Dimitri
run applications, eg. python -m SimpleHTTPServer, generate.py
source-server, editor/IDE embedded etc. This is because of limitations
of
Post by Adrian Haarbach
Post by Dimitri
file:/// addressing scheme and CORS/XHR issues that it has.
QxCompiler is going to introduce another background process for live
code
Post by Adrian Haarbach
Post by Dimitri
transpiling. Why not combining these two, so that the developer doesn't
have to launch two separate processes?
It should be pretty easy to serve static contents from Node. In the
future, automatic application reloading could be implemented, something
that is available now with generate.py watch + source-server-reload
jobs.
Post by Adrian Haarbach
Post by Dimitri
Dimitri
Hi all
There is a first release of my QxCompiler that adds ES6 to Qooxdoo
applications and replaces the generate.py toolchain with a faster, 100%
Javascript tool that is easily extensible.
https://github.com/johnspackman/qxcompiler
It’s an alpha release, but something that’s been in development for a
while now and which I’m starting to build into my production servers.
I’m very open to pull requests or collaboration, and keen to see this
become a useful tool for myself as well as others so any questions etc
please ask.
Regards
John
------------------------------------------------------------------------------
Post by Adrian Haarbach
Post by Dimitri
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
------------------------------------------------------------------------------
Post by Adrian Haarbach
Post by Dimitri
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
------------------------------------------------------------------------------
Post by Adrian Haarbach
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
d***@cost-savers.net
2016-02-22 21:10:36 UTC
Permalink
Thomas,
Post by thron7
I could see Grunt being a command-line frontend for the jobs, with
QxCompiler remaining a pure API-based library which is included as a
dependency and utilized from Grunt files.
I agree completely with you in this! Don't invent the wheel again!
Post by thron7
Even if you feel the code in qooxdoo.next is too 'Grunt-heavy', or would
just like to hack on an all-new, own version of Grunt support (which I can
always understand ;-)), the code in qooxdoo.next might be a good
inspiration to draw from.
No new hack. Reuse what already exists because it is good for what it was done. I have looked at it. It can be used here.

Stefan
Post by thron7
T.
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
John Spackman
2016-02-23 07:22:52 UTC
Permalink
Thomas, that’s interesting about the grunt development in next, I had no idea it had gone so far.

Stefan, I’ve been thinking about not reinventing the wheel, esp. in light of Thomas’ comments, and my thought is that QxCompiler should be able to produce an application from source, from the command line and the use of Grunt (or Gulp, or brocolli, etc) should only be required for more advanced, custom tasks.

My rationale for this is that QxCompiler only needs a simple command line in order to achieve this (for new and experienced users), and does not require any kind of third-party build tool with all the learning that is required to get started with one.

One thing I would recommend is to look at dropping the config.json support altogether (and Python); with the compiler functionality being replaced by QxCompiler, most of what’s left is definitely in the area of grunt/gulp/etc; eg generating API docs, generating complete demo apps, distribution kits, etc.

Although I can have a look at Grunt, I have no experience with it and my focus is on QxCompiler at the moment; briefly looking through the next master/tool/grunt repo, the source looks interesting but will probably take some time to get to grips with Grunt to really use.

Is this something you could help with? Like QxCompiler, it could be developed as a separate contrib and I’d be happy to help with parts of the QxCompiler API to support you.

Regards
John
Post by d***@cost-savers.net
Thomas,
Post by thron7
I could see Grunt being a command-line frontend for the jobs, with
QxCompiler remaining a pure API-based library which is included as a
dependency and utilized from Grunt files.
I agree completely with you in this! Don't invent the wheel again!
Post by thron7
Even if you feel the code in qooxdoo.next is too 'Grunt-heavy', or would
just like to hack on an all-new, own version of Grunt support (which I can
always understand ;-)), the code in qooxdoo.next might be a good
inspiration to draw from.
No new hack. Reuse what already exists because it is good for what it was done. I have looked at it. It can be used here.
Stefan
Post by thron7
T.
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
d***@cost-savers.net
2016-02-23 11:33:43 UTC
Permalink
John,
Post by John Spackman
Thomas, that’s interesting about the grunt development in next, I had no idea it had gone so far.
Stefan, I’ve been thinking about not reinventing the wheel, esp. in light of Thomas’ comments, and my thought is that QxCompiler should be able to produce an application from source, from the command line and the use of Grunt (or Gulp, or brocolli, etc) should only be required for more advanced, custom tasks.
My rationale for this is that QxCompiler only needs a simple command line in order to achieve this (for new and experienced users), and does not require any kind of third-party build tool with all the learning that is required to get started with one.
I agree, and I look at it as Thomas, qxcompiler is a library, which can be activated by a small commandline interface or through the whole Builder/Generate.
Post by John Spackman
One thing I would recommend is to look at dropping the config.json support altogether (and Python); with the compiler functionality being replaced by QxCompiler, most of what’s left is definitely in the area of grunt/gulp/etc; eg generating API docs, generating complete demo apps, distribution kits, etc.
Yes, for the compiler with a good API activating the same functionality at will, but not for the whole builder where config.json becomes like a Makefile and makes things easier that way.
Post by John Spackman
Although I can have a look at Grunt, I have no experience with it and my focus is on QxCompiler at the moment; briefly looking through the next master/tool/grunt repo, the source looks interesting but will probably take some time to get to grips with Grunt to really use.
Is this something you could help with? Like QxCompiler, it could be developed as a separate contrib and I’d be happy to help with parts of the QxCompiler API to support you.
We can look at it and try to replace all python with grunt integration further on the line the core team did before.
I can back on this asap.

Stefan
Post by John Spackman
Regards
John
Post by d***@cost-savers.net
Thomas,
Post by thron7
I could see Grunt being a command-line frontend for the jobs, with
QxCompiler remaining a pure API-based library which is included as a
dependency and utilized from Grunt files.
I agree completely with you in this! Don't invent the wheel again!
Post by thron7
Even if you feel the code in qooxdoo.next is too 'Grunt-heavy', or would
just like to hack on an all-new, own version of Grunt support (which I can
always understand ;-)), the code in qooxdoo.next might be a good
inspiration to draw from.
No new hack. Reuse what already exists because it is good for what it was done. I have looked at it. It can be used here.
Stefan
Post by thron7
T.
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
John Spackman
2016-02-23 11:55:40 UTC
Permalink
Post by d***@cost-savers.net
We can look at it and try to replace all python with grunt integration further on the line the core team did before.
I can back on this asap.
Stefan
Fantastic :)

Cheers
John
d***@cost-savers.net
2016-02-25 13:42:54 UTC
Permalink
Hi Defero,

You need to check the following for dynamic loading:

1. if the part has already been loaded (version, build, date etc)
- different versions of the same part might be loadable at the same time
- different language parts
- etc
2. if there are cross referencing
3. circle referencing
4. the integrity ...the questions is what is necessary...?
5. some kind of lazy loading to preserve memory and decrease loading time

...a list at runtime...how should it else be done?

Stefan
Post by Dietrich Streifert
Hi John,
so i've been thinking how i would start creating the part loader which
should be "easy" to create with how you set up the things.
But for it to fit our purpose here, it asks that only a single part/library
can be recompiled.
I'm not sure if there is a nice way of achieving this. In case of two parts
have the same dependency but are not aware of each other. Which in most UI
qx classes i think shouldn't be an issue if they are compiled/loaded twice.
But i'm sure there are plenty of classes that have to strictly be loaded
once onto dom.
Do you think there is a way to achieve this without cross-referencing every
part and adding to boot the same dependencies.
Regards,
Defero
--
View this message in context: http://qooxdoo.678.n2.nabble.com/QxCompiler-add-ES6-faster-compilation-and-100-Javascript-API-to-building-applications-tp7587992p7588112.html
Sent from the qooxdoo mailing list archive at Nabble.com.
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
qooxdoo-devel mailing list
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
Loading...