-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathScholarly HTML technical report.html
More file actions
750 lines (741 loc) · 50.3 KB
/
Scholarly HTML technical report.html
File metadata and controls
750 lines (741 loc) · 50.3 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes">
<title>WebSys Project Final</title>
<link rel="stylesheet" href="scholarly.min.css">
</head>
<body prefix="schema: http://schema.org">
<header>
<div id="top">WebSys Project - Design</div>
<img id="logo-center" src="logo.png" alt="ShoppingList logo">
<div id="top-ending">end of label</div>
</header>
<hr>
<div role="contentinfo">
<h2>Table of Contents</h2>
<ol role="directory">
<li><a href="#abstract"><span>1. </span>Abstract</a></li>
<li><a href="#introduction"><span>2. </span>Introduction</a></li>
<li>
<a href="#structure"><span>3. </span>Structure</a>
<ol role="directory">
<li><a href="#Design"><span>3.1 </span>Overall Design</a></li>
<li><a href="#Pinpoints"><span>3.2 </span>Pinpoints</a></li>
<ol role="directory">
<li><a href="#UntangleWebCache"><span>3.2.1 </span>Untangle Web Cache</a></li>
</ol>
<ol role="directory">
<li><a href="#NginxTSL"><span>3.2.2 </span>TSL implementation with Nginx</a></li>
</ol>
<ol role="directory">
<li><a href="#AWSLoadBalancer"><span>3.2.3 </span>AWS Load Balancer</a></li>
</ol>
<ol role="directory">
<li><a href="#JBoss"><span>3.2.4 </span>JBoss Enterprise Application Platform</a></li>
</ol>
<ol role="directory">
<li><a href="#Solr"><span>3.2.5 </span>Solr</a></li>
</ol>
<ol role="directory">
<li><a href="#Oracle"><span>3.2.6 </span>Oracle Database</a></li>
</ol>
<ol role="directory">
<li><a href="#Spark"><span>3.2.7 </span>Apache Spark</a></li>
</ol>
<ol role="directory">
<li><a href="#ApacheNutch"><span>3.2.8 </span>Apache Nutch</a></li>
</ol>
<ol role="directory">
<li><a href="#Retailers"><span>3.2.9 </span>Retailers</a></li>
</ol>
<ol role="directory">
<li><a href="#CDN"><span>3.2.10 </span>CDN</a></li>
</ol>
<ol role="directory">
<li><a href="#Firewall"><span>3.2.11 </span>Firewall</a></li>
</ol>
<li>
<a href="#FirstModule"><span>3.3 </span>First Module</a>
</li>
<li><a href="#SecondModule"><span>3.4 </span>Second Module</a></li>
<li><a href="#ThirdPartyAPI"><span>3.5 </span>Third Party APIs</a>
<ol role="directory">
<li><a href="#AmazonPay"><span>3.5.1 </span>Amazon Pay</a></li>
<li><a href="#FacebookAPI"><span>3.5.2 </span>Facebook Development API</a></li>
<li><a href="#GoogleMaps"><span>3.5.3 </span>Google Maps</a></li>
<li><a href="#GoogleAnalytics"><span>3.5.4 </span>Google Analytics</a></li>
</ol>
</li>
<li><a href="#Flows"><span>3.6 </span>Architectural Considerations</a>
<ol role="directory">
<li><a href="#IOFormat"><span>3.6.1 </span>Input/Output data formats</a></li>
<li><a href="#DataFlow"><span>3.6.2 </span>Data Flow</a></li>
<li><a href="#TaskFlow"><span>3.6.2 </span>Task Flow</a></li>
</ol>
</li>
</ol>
</li>
<li>
<a href="#Interface"><span>4. </span>Interface</a>
<ol role="directory">
<li><a href="#Main"><span>4.1 </span>Main Page</a></li>
<li><a href="#Modules"><span>4.2 </span>Main Modules</a>
<ol role="directory">
<li><a href="#ModuleOneDesign"><span>4.3.1 </span>First Module</a></li>
<li><a href="#ModuleTwoDesign"><span>4.3.2 </span>Second Module</a></li>
</ol>
</li>
<li><a href="#LogoC"><span>4.4 </span>Logo</a></li>
</ol>
</li>
<li><a href="#UserGuide"><span>5. </span>User Guide</a>
<ol role="directory">
<li><a href="#CaseStudy"><span>5.1 </span>Case Study</a>
<ol role="directory">
<li><a href="#CaseStudyNr1"><span>5.1.1 </span>Return a product</a></li>
<li><a href="#CaseStudyNr2"><span>5.1.2 </span>Ask for warranty</a></li>
<li><a href="#CaseStudyNr3"><span>5.1.3 </span>Place a instant purchase</a></li>
</ol>
</li>
<li><a href="#VideoDemonstration"><span>5.2 </span>Video Demonstration</a></li>
</ol>
</li>
<li><a href="#bibliography"><span>6. </span>Bibliography</a></li>
<li><a href="#Annexes"><span>7. </span>Annexes</a></li>
</ol>
<dl>
<dt>Authors</dt>
<dd>
Marius-Iulian Cluci
<br />
<a href="https://www.linkedin.com/in/vasilica-mihai-horgot-b8ab02a6/?originalSubdomain=ro">Vasilica Mihai Horgot</a>
<br />
<a href="https://www.linkedin.com/in/alexandra-alina-pavel-856278b0/?originalSubdomain=ro">Alexandra Alina Pavel</a>
</dd>
<dt>Bugs & Feedback</dt>
<dd>
<a href="https://github.com/w3c/scholarly-html/issues">Issues and PRs welcome!</a>
</dd>
<dt>Issues and Discussions</dt>
<dd>
Our <a href="https://github.com/mcluci/WebSysDevelopment---Automatic-Shopping-List/issues">Team WebSys Project</a>
at <a href="https://github.com">github</a>
(<a href="https://github.com/mcluci/WebSysDevelopment---Automatic-Shopping-List/wiki/Contact">or email us</a>)
</dd>
<dt>License</dt>
<dd>
<a href="http://creativecommons.org/licenses/by/4.0/">CC-BY</a>
</dd>
</dl>
</div>
<section typeof="sa:Abstract" id="abstract" role="doc-abstract">
<h2>Abstract</h2>
<p>
The following technical report will present the design of a web application made
to search and follow prices of all popular e-commerce websites in order to find
the best deals at best price.
In the following pages we will present the technical design, wanted features,
options, modules and the third-party APIs we will incorporate in our application.
Our web app will try to automatically buy certain goods (electronics and other items)
with certain characteristics and price set by the user from different shops in the
next 30 to 60 days time frame, using crawlers and other APIs to accomplish its task.
</p>
</section>
<section id="introduction" role="doc-introduction">
<!-- review? -->
<h2>Introduction</h2>
<p>
Nowadays we all buy goods online it is easy, fast and comfortable, but that doesn't mean
that we won't face issues, considering that we might have all sort of wants, like the color,
the brand, the model, and the technical characteristics of the product we might get a big headache
when we want to buy a certain thing.
When you add the fact that there are many online retailers, things start to complicate a lot!
According to <a href="https://www.statista.com/statistics/568735/e-commerce-website-visit-duration/">Statista</a>
in the third quarter of 2018 an average shopper spent approximatively 4 minutes on a website visit
multiply that with the number of retailers visited and then you might get an approximate of how much
we spend on online shopping. Can't we reduce this?
Not to mention that the statistics show that in just 2.86 percent of e-commerce website visits are
converted into purchases, as statistics <a href="https://www.statista.com/statistics/439576/online-shopper-conversion-rate-worldwide/">show.</a>
meaning that a lot of time is wasted on just searching products.
</p>
<p>
Although the price might not be the most important feature, we could rank it in top 3 most important
features in buying an item online. Thus, offering the smallest price to a customer is imperatively important.
This is why we design the following app, in order to simplify things for the average shopper by
automating the purchase of certain goods, at a certain set price, or (if they want to buy on spot) at the
minimum price currently available on the market.
</p>
<p>
Thus, the following pages will describe ShoppingList, an application meant to automatically buy
wanted product at wanted prices using crawlers, non-sql databases and third-party payment processors.
We assumed 2 somehow similar main functionalities (modules) of this website. The first module will
readily be available for our users to search and buy product at best prices from different retailers,
and the second module will allow users to buy at a better price buy placing a batch job which for the next 30 to
60 days will scope the prices for that specific product in order to buy it at a price lower (or equal) to the one
set by the user.
</p>
<p>
Thus, the first module will refer to user willing to buy product as soon as possible
whilst the second module will be oriented to users willing to buy at a certain price and for whom
the waiting time is not as important. The second module might also refer to companies which could buy in bulk
at a much smaller price.
</p>
<p>
Our main website functions are:
</p>
<ul>
<li>Allow users to search products and find the retailer with the smallest price.</li>
<li>Allow users to compare prices across retailers.</li>
<li>Allow users to create batch jobs that will automatically buy products at a set price in the following 30 to 60 days.</li>
<li>See a graph with changes in prices across retailers.</li>
<li>Allow users to set as much wanted characteristics to a automatic batch purchase.</li>
</ul>
</section>
<section id="structure">
<!-- Whole Sys Structure -->
<h2>Structure</h2>
<p>
The internal design of our app is described in the <a href="#Design">design section</a> bellow, there you
will also find an detailed exemplification of how the components will interact between each other.
In this same section you will also find the two modules fully described with their main functionalities
and their core. Also all the third-party APIs used in the project will we described below, with the main emphasize
on the payment processor.
</p>
<section id="Design">
<!-- Design -->
<h3>Overall Design</h3>
<p>
The figure below describes our main application design on a higher level. You will find some red numbered dots
in it, those <a href="#Pinpoints">pinpoints</a> are later explained below in detail, for a better understanding of our ideas. The web application
will use multiple software solutions in order to fully function. What you will find different is the way in which the crawler fetches data
from some online retailers in order to allow Apache Spark perform its task, and if necessary create the buy order.
</p>
<figure typeof="sa:image" id="MainDesignFigure">
<img src="images/MainDesignPoints2.png" alt="Main application design">
<figcaption>Figure 1. Main Design</figcaption>
</figure>
</section>
<section id="Pinpoints">
<!-- Main Components -->
<h3>Design Pinpoints</h3>
<p>
In the following row we will present the main pinpoints that appear in the figure from the <a href="#Design">design section</a>:
</p>
<section id="UntangleWebCache">
<h4><img class="pinpoint" src="images/pin1.png" alt="Pinpoint number 1."> Untangle Web Cache</h4>
<p>
Once the user makes the https request to the server, <a href="https://www.untangle.com/">Untangle</a> will act as a
web cache to reduce server lag and load, by providing already stored resources (either html, images or any other site asset) to
the user at a greater speed and less expense. Untangle has great performance and it should be granted several servers in order
to have enough resources to serve cached objects. Untangle is also a firewall, but we won't use it in this case, as pricing goes high
and we are only interested into its high performance caching. Anyway, being a multi-function tool it offers great compatibility with other
security related applications.
</p>
</section>
<section id="NginxTSL">
<h4><img class="pinpoint" src="images/pin2.png" alt="Pinpoint number 2."> TSL implementation with Nginx</h4>
<p>
Transport Layer Security (TLS) is the successor to Secure Socket Layer (SSL). It provides stronger and more efficient HTTPS, and contains enhancements not found in SSL such as Forward Secrecy, compatibility with modern OpenSSL cipher suites, and HSTS. Having this set in Nginx
will secure upstream servers and ensure client data is safe.
</p>
<section>
<section id="AWSLoadBalancer">
<h4><img class="pinpoint" src="images/pin3.png" alt="Pinpoint number 3."> AWS Load Balancer</h4>
<p>
Using <a href="https://aws.amazon.com/elasticloadbalancing/features/?nc=sn&loc=2">AWS Elastic Load Balancer</a> will ensure great workload distribution among the main servers, both for hardware resources and network. Its Network Load Balancer is best suited for load balancing of TCP traffic where extreme performance is required. Operating at the connection level, Network Load Balancer routes traffic to targets within Amazon Virtual Private Cloud (Amazon VPC) and is capable of handling millions of requests per second while maintaining ultra-low latencies. Network Load Balancer is also optimized to handle sudden and volatile traffic patterns, if needed.
A mention well note, we will use a hybrid cloud services and run most of our servers in AWS, as it also offer great <a href="https://aws.amazon.com/oracle/">support</a> for oracle.
</p>
</section>
<section id="JBoss">
<h4><img class="pinpoint" src="images/pin4.png" alt="Pinpoint number 4."> Main servers - JBoss Enterprise Application Platform</h4>
<p>
JBoss EAP is a subscription-based/open-source Java EE-based application server runtime platform used for building, deploying, and hosting highly-transactional Java applications and services.The JBoss Enterprise Application Platform is part of the JBoss Enterprise Middleware portfolio of software. Because it is Java-based, the JBoss application server operates across platforms; it can easily run on any operating system that supports Java and has great flexibility.
We motivate our choice also due to the fact that it has great compatibility and integration with Java based applications, which will mainly
be the core of our web application. In plus, the subscription applies once you go in production, using these servers, which might cut costs.
We might also use <a href="https://apacheignite.readme.io/v2.0/docs/memory-and-cache-metrics">Apache Ignite</a> to accelerate existing Relational and NoSQL databases with Ignite™ in-memory data grid and <a href="https://aws.amazon.com/elasticloadbalancing/features/?nc=sn&loc=2">caching capabilities</a>
</p>
</section>
<section id="Solr">
<h4><img class="pinpoint" src="images/pin5.png" alt="Pinpoint number 5."> Solr for search mechanism</h4>
<p>
As our products have a lot of features and characteristics, a good search engine is required. We have chosen Solr, due to its capabilities to
do full-text search, hit highlighting, faceted search, real-time indexing, dynamic clustering, database integration and rich document (e.g., Word, PDF) handling.
Solr is very used in the market, and offers great support and flexibility being also free.
</p>
</section>
<section id="Oracle">
<h4><img class="pinpoint" src="images/pin6.png" alt="Pinpoint number 6."> Oracle Database for transactions</h4>
<p>
NoSql databases might be nice, fast and flexible, but most of them lack support for transactions, and as our application will make such
transaction (purchase, refunds and others) it is highly important to use Oracle Database in order to ensure full transaction support (to ensure rollback, commit and other operation).
All Oracle transactions obey the basic properties of a database transaction, known as ACID properties. Properties highly important when we speak
of financial transactions, which are sensitive to us, to our customers and in front of the law.
</p>
</section>
<section id="Spark">
<h4><img class="pinpoint" src="images/pin7.png" alt="Pinpoint number 7."> Apache Spark for product history</h4>
<p>
Apache Spark is a fast in memory database (can also use just HDD is wanted) that could contain large databases, well over a petabyte.
This database will be use to store product prices and other characteristics that might just not fit our RDBMS (or have a huge performance impact).
Thus the large amounts of data feed by our crawlers will be handled by Spark, where some task will also be performed in order to ensure one of our main functionality (automated buying).
Furthermore, Spark supports streaming and machine learning alghorithms, which might prove handy in our case, or for future development. It is also free, can run in most cloud environments, and it has good compatibility with other systems.
</p>
</section>
<section id="ApacheNutch">
<h4><img class="pinpoint" src="images/pin8.png" alt="Pinpoint number 8."> Apache Nutch Crawler</h4>
<p>
As we will be scanning many retailer website, we will focus on the usage of crawlers. As we have a lot of apache product in our design
we decided to go for Apache nutch. Apache Nutch is a highly extensible and scalable open source web crawler software project.
The crawler will take all the information for all the list of products from the retailers, and send it to our Spark Database, where it will be
stored, compare and matched with other characteristics.
According to a <a href="http://www.cecs.uci.edu/~papers/ipdps07/pdfs/SMTPS-201-paper-1.pdf">IBM study</a> Apache Nutch clustering can get
awesome performance, yet to be achieve by other crawlers and systems.
</p>
</section>
<section id="Retailers">
<h4><img class="pinpoint" src="images/pin9.png" alt="Pinpoint number 9."> Retailers</h4>
<p>
Retailers websites and RSS is vital for our application, to avoid DDos false accusations, we will do our best to partnership or use as less
invasive methods to gather data with the crawlers.
Whether it is fully possible or not, it is to be seen in the future.
</p>
</section>
<section id="CDN">
<h4><img class="pinpoint" src="images/pin10.png" alt="Pinpoint number 10."> Content delivery network for performance</h4>
<p>
Amazon CloudFront is a content delivery network (CDN) made and delivered by Amazon through its services, on a pay as you go philosophy. Content delivery networks provide a globally-distributed network of proxy servers which cache content, such as web videos or other bulky media, more locally to consumers, thus improving access speed for downloading the content, which will allow better SEO and SEO ranking, not to mention less load on our servers which won't have to deliver performance costly images or video.
CloudFront has servers all around the globe in all continents, which offers great performance and support. It is a bit costly though!
</p>
</section>
<section id="Firewall">
<h4><img class="pinpoint" src="images/pin11.png" alt="Pinpoint number 11."> Firewall</h4>
<p>
Although not much control is given over our cloud hosted environments, a hybrid could environment does have some local machines, and we find appropriate to use Barracuda Firewall, due to its dual support, for both hardware and software.
Cloud firewall and security measures will also be used.
</p>
</section>
</section>
<section id="FirstModule">
<!-- Modules -->
<h3>First Module - Price comparison</h3>
<p>
One of our most important function, is to provide our users with products having the lowest price on the market from all the specified retailers.
Giving fast responses in this case would require to have the data already in the database (crawlers will take care of this at certain intervals
of time). Historical data is as important as any other product characteristics, and we plan on giving a graph with historical prices from
all the scoped retailers.
Custom filtering on this search would be essential, Solr and Apache Spark will be responsible for this process.
</p>
<p>
The first part is represented by an advanced search, that will perform full-text search in our product database and list all available products.
In the image from below you will see a list of different products. Only after clicking on one, you will be able to see a comparison of prices among retailers.
In this page the price displayed is the lowest at the moment.
</p>
<p>
<figure typeof="sa:image" id="searchProduct">
<img src="images/SearchProduct.png" alt="Search page and characteristics">
<figcaption>Figure 2. Search Page Sketch</figcaption>
</figure>
</p>
<p>
As main functionalities, we have to highlight the left widget, which will allow good filtering options, according to any wanted characteristics, such as price, brand, color
and many more, depending by the product.
The different models of the specific product will be listed.
</p>
<p>
Once a product is chosen, and <i>See offers</i> is clicked, the user will be redirected to the <i>Product Details Page</i> which is described below in the Figure 3.
This page will have many functionalities, with its main being to display a list of prices sorted by each big retailer.
</p>
<p>
<figure typeof="sa:image" id="productDetails">
<img src="images/productDetail.png" alt="Product Details Page">
<figcaption>Figure 3. Product Details Page Sketch</figcaption>
</figure>
This page will include product images, product description, user comments and review, plus a graph containing the price history of the specific product for a certain period of time.
All the retailers owning such a product will be displayed bellow, together with their catalog price. Further sorting is allowed according to the price and retailer.
What we also integrated is the <i>Add to batch</i> which will create a batch order of automatic purchase, and will link module 1 to the <a href="#SecondModule">second module</a> which is described below.
</p>
</section>
<section id="SecondModule">
<!-- Modules -->
<h3>Second Module - Automatic Purchases</h3>
<p>
Our second module, or core functionality is to allow the user to set automatic purchases of certain items at a fixed (or a range) of prices. The so called <i> automatic batch purchases</i> will represent one of our main competitive advantage. The use of third-party payment APIs and services will be essential in this case, as it will be very complicated to build such systems from scratch, not to mention the security risks and bugs involved, and the huge cost.Refer to <a href="#ThirdPartyAPI">APIs</a> for more.
The main functionalities of this core module are displayed below:
<ul>
<li>
Automatic buying in the future (from 30 to 60 days);
</li>
<li>
Allow multiple items purchase in a <i>batch purchase</i> and combination of items;
</li>
<li>
Matching between wanted product features and the catalog of products from the retailers (tricky to do);
</li>
<li>
Include some logic in the buy orders (example: <code>If product IS green OR black THEN buy ELSE stop;</code>);
</li>
<li>
Use a payment processors to take the money from the client account and keep those money blocked until the product is bought, or the automatic purchase expires;
</li>
<li>
Allow SMS notifications and SMS buy approval <i>queries</i> from the user. (the user will receive a SMS in which he will be ask if he is willing to buy or not);
</li>
</ul>
</p>
</section>
<section id="ThirdPartyAPI">
<!-- APIs -->
<h3>Third Party APIs</h3>
<p>
The usage of third part APIs is mainly motivated by some advantages and cost reductions provided by them, such as the payment processor which increases security and reduces risk.
In the rows below we will describe all the third party APIs we intent to use.
<figure typeof="sa:image" id="externalAPIs">
<img src="images/DesignWithAPIs.png" alt="API Design">
<figcaption>Figure 4. Public APIs</figcaption>
</figure>
Although not included in the <a href="#MainDesignFigure">main aplication design</a>, the figure above refers to the way in which our services will connect to external public APIs.
These APIs are marked in the picture with orange connectors.
</p>
<section id="AmazonPay">
<h4>Amazon Pay</h4>
<p>
As a payment processor, Amazon pay offers great features, and can be either very easy to manage or, can be the perfect tool for a expert in finance and marketing, depending by your choice.
It has multi-platform support, and can be integrated in almost any app. Amazon Pay delivers a seamless checkout experience across desktop, mobile, and voice – making it easy for customers to make purchases wherever they are. It has great security and advanced encryption, an also it gives you certain insurances. Amazon Pay’s fast & easy checkout process, built on years of Amazon’s innovation in checkout optimization can help increase conversions & lower cart abandonments for buyers using their existing Amazon accounts. It also has fraud detection mechanisms and offers great support to you, and to your customers.
It also offers easy monitoring, cost reductions and great performance with low latency and all around the glob distributed servers. It is one of the most important APIs which offers a great <a href="https://aws.amazon.com/api-gateway/?sc_channel=PS&sc_campaign=acquisition_RO&sc_publisher=google&sc_medium=api_gateway_b&sc_content=api_bmm&sc_detail=%2Bamazon%20%2Bapi&sc_category=api_gateway&sc_segment=158526582628&sc_matchtype=b&sc_country=RO&s_kwcid=AL!4422!3!158526582628!b!!g!!%2Bamazon%20%2Bapi&ef_id=EAIaIQobChMI043sjceN3wIVycmyCh2dXwRJEAAYASAAEgLdG_D_BwE:G:s">palete of features</a>.
</p>
<p>
The way in which Amazon Pay will connect can be see in <a href="#externalAPIs"> API Design</a>.
</p>
</section>
<section id="FacebookAPI">
<h4>Facebook Development API</h4>
<p>
Registring or login with facebook is a popular mean of account control these days, Facebook API is require to allow this connectivity, thus we must include it. Facebook API also offers great support for sharing pages, which is a great way in which we might gain more visits from social networks.
</p>
</section>
<section id="GoogleMaps">
<h4>GoogleMaps</h4>
<p>
This will allow the users to set a point to where the courier will deliver, or it will direct them to a physical shop or pick-up point.
</p>
</section>
<section id="GoogleAnalytics">
<h4>GoogleAnalytics</h4>
<p>
Google Analytics is now the most widely used web analytics service on the web. Google Analytics also provides an SDK that allows gathering usage data from iOS and Android Apps, known as Google Analytics for Mobile Apps.
</p>
</section>
<p>This Concludes our list of public APIs,</p>
</section>
<section id="Flows">
<!-- Flows / Task / Data -->
<h2>Architectural Considerations</h2>
<p>
The parts presented below want to bring more considerations in the eyes of the viewer (experimented or novice) so that he would further understand the full concept.
</p>
<section id="IOFormat">
<h3>Input/Output data formats</h3>
<p>
The data sent and receive between the client, server and externals is of various formats. The data transferred between the server and the user is protected using different protocols and encryptions. The user is able to send and receive plain data in the html pages.
The data exchanged between the server and the external retailers will be in Json format (our main application will be written in Java also) due to Json high customizations and flexibility. Furthermore, many crawlers work best in outputting such formats which allow better support for database categorization.
The third party APIs (such as Amazon Pay) will offer certain web services that provide data in many formats, but mainly XML or Json for larger transactions and data imports.
</p>
</section>
<section id="DataFlow">
<h3>Data Flow</h3>
<p>
The main data flow of our most important module is presented in the figure below. The Diagram has been made in accordance
to Gane & Sarson rules and figures and depicts the flow of data in our aplication. In the rows below we take the main parts
of the diagram and discuss them in-depth for further clarifications.
<figure typeof="sa:image" id="DataFlowDiag">
<img src="images/DataFLow.png" alt="Data Flow Diagram">
<figcaption>Figure 5. Data Flow</figcaption>
</figure>
</p>
<h4><img class="pinpoint" src="images/pin13.png" alt="Pinpoint number 13."> User Login</h4>
<p>
In order to make a order the user must have a account with all the required informations (delivery address, phone number,
and other relevant details). Thus he must first log in with a valid account.
The data flown between the server and the user is secure with TSL and https, in order to protect the information passed from
the user to the server, and vice versa.
The user will just pass the account email / or pone number and password to enter its account. The server will only reply with a
confirmation or rejection of credentials. Later on, the user could change the account details from the account settings page.
</p>
<h4><img class="pinpoint" src="images/pin14.png" alt="Pinpoint number 14."> Add purchase plan (or batch)</h4>
<p>
Now, the client can plan a automatic future purchase of a wanted product, set the required product details, preferred retailers
and set a price limit. All this information is processed by our application (within the process called in our diagram "Planned
Purchase") and the specfications set by our client is inserted in a table where it will stay according to the client settings (30 to 60 days accordingly).
This process ensures that all the set data is correct and the plan is prepared for the other processes which will try to match
products according to the specified details and price.
</p>
<h4><img class="pinpoint" src="images/pin15.png" alt="Pinpoint number 15."> Funds Collection</h4>
<p>
Considering that we seek the smallest prices and offers, we must be quick in buying the products, thus the pay is asked in advance. In this way, we will be sure to have the required money before buying. But, considering that it is very ricky to work with such secure money transfers, and the development will require plenty resources, we decided to use a third-party API named Amazon Pay which will ensure money transfer and refunds.
Keep in mind that if there is no product purchase the money will be transferred back within the required number of days (the client supports the transfer charges).
The data which is transferred in this step is very simple, the client is redirected with an id to the third-party API, where the data transferred there is not of our interest, but the API will get back to use with a response in which will specify if the transfer is successful or not. After this the planned buy order is added in the batch processing, where it could conclude in a purchase.
Further data will be requested, and other operations might be required, such as refunds.
</p>
<h4><img class="pinpoint" src="images/pin16.png" alt="Pinpoint number 16."> Daily Price Comparation</h4>
<p>
If for the process 1, 2 and 3 the data is transferred between the client and the server, at process number 4 the data is transferred between the server and other retailers with the help of a crawler (and some RSS informations).
The crawler will gather thousands of product details and prices, and will classify and insert those information in the database. When new data is inserted for some products, specific triggers will order a price comparisson for those
Thus, our application will run at constant time making comparisons between the prices (& details) and the planned purchase orders to ensure that we don't miss an offer for our clients and make as much purchases as possible.
</p>
<h4><img class="pinpoint" src="images/pin17.png" alt="Pinpoint number 17."> Crawlers Gather Data</h4>
<p>
As better described in process 4, the Crawler(s) will gather data from the retailers, either by analyzing the RSS feed or by using (if available) special pages (that are subject of a written cooperation between us and the retail). Much of the data is transferred between the server and the retailer.
The collected data will also be categorized in this step for best practices and performance.
</p>
<h4><img class="pinpoint" src="images/pin18.png" alt="Pinpoint number 18."> Return Scenario</h4>
<p>
It is quite clear that not all planned purchases will be accomplished, thus money should be returned to the client.
I this case data is exchanged between us and the Amazon Pay API in order to order to money refund. Data will also be exchanged between in and the client in the matter of a email or website posting, in which we inform the user of our actions.
In the case in which the client had the purchase done, but he wants to return the product from various reasons, the scenario might be a bit more complicated, as data needs to be transferred to the retailers also. The request will be automatically processes, in order to ensure the product return. The money will get back straight to the client account, in order to avoid Amazon Pay fees.
</p>
</section>
<section id="TaskFlow">
<h3>TaskFlow</h3>
<p>
The taskflow presented below (<a href="#TaskFlowDiag">Figure 6</a>) resembles the path followed by the user that wants to add a planned order. This visual summary of broad types of tasks which is then broken down into series of smaller tasks to have a mapping of the related tasks so these tasks may then be executed easily. It provides a graphically represented order of the tasks to be performed from beginning until the end.
<figure typeof="sa:image" id="TaskFlowDiag">
<img src="images/TaskFLow.png" alt="Data Flow Diagram">
<figcaption>Figure 6. Task Flow</figcaption>
</figure>
</p>
</section>
<p>
This Concludes our further architectural discussions. The above <a href="#DataFlow">Data flow</a> and <a href="#TaskFlow">Task flow</a> should bring more light in the eyes of the viewer, and furher understand the full extent of our architecture.
</p>
</section>
</section>
</section>
</section>
<section id="Interface">
<h2>Interface</h2>
<section id="Main">
<h3>Main page</h3>
<p>
Our Interface will try to be as good as possible and to offer great compatibility across devices and be easy to read and use.
The image below shows a sketch of our homepage, we also used some images in order to try to make it look like a prototype.
<img src="images/main.png" alt="App Homepage">
</p>
Nothing much different than any retailer website, except for the two main modules, we described above. In the menu of our app, you will see <i>Pending Orders</i> and <i>Purchases</i> both having notifications, meaning that:
<ul>
<li>
Considering the green dot notification with number 1, we have a successful automatic purchase, we can see in <a href="#autoPurchase">Automatic Purchase Page</a>
</li>
<li>
Considering the redd dot notification having number 2, we have 2 unsuccesfull or pending automatic purchases, we can see in <a href="#batchPurchaseStats">Automatic Purchase Order</a> page.
This would mean that the price we set (or the range of prices) for a purchase order was not yet reach, and that the wanted product with its characteristics is still more expensive and we have to way for the price to drop in order to satisfy the set criterias.
</li>
</ul>
</section>
<section id="Modules">
<h3>Main Modules</h3>
<section id="ModuleOneDesign">
<h4>First Module</h4>
<p>Please refer to the structure design to see the first module.</p>
<ul>
<li>
<a href="#productDetails">Product Details and comparison page</a>
</li>
<li>
<a href="#searchProduct">Product Search page and its options</a>
</li>
</ul>
</section>
<section id="ModuleTwoDesign">
<h4>Second Module</h4>
<p>
The first part will include the order creation form, as shown in the image below:
<img src="images/createBatchPurchase.png" alt="batch Creation Form image 1">
It has 5 sections all drop-down, each having a scope. The first section is for chosing the products, setting its characteristics and features.
The second will allow you to set a price or a price range, according to your wishes.
The third, will allow you to chose from the retailers. Filtering by your wish.
Custom Rules is also added, but it is optional.
The last Section will redirect to Amazon Payments.
</p>
<p>Part 1:</p>
<img src="images/batch1.png" alt="batch Creation Form image 2">
<p>Part 2:</p>
<img src="images/batch2.png" alt="batch Creation Form image 3">
<p>Part 2:</p>
<img src="images/batch3.png" alt="batch Creation Form image 3">
</section>
</section>
<section id="LogoC">
<h3>Logo</h3>
<p>
Our chosen logo is the following:
</p>
<img src="Logo.png" alt="Company Logo">
<p>
It has <i>mild</i> colors and it resembels its main functionality, that of making purchases by its own. We have chosen a nice font for the name of the product. We also wanted to add the slogan(<i>Best place for best deals.</i>), but that would make it to large. We will use the slogan on presentation pages. The logo is just enough for our company. We can also use the logo as a favicon, the <i>S</i> will fit nicely in the tab header of the browser.
Your brand strategy is how, what, where, when and to whom you plan on communicating and delivering on your brand messages. Where you advertise is part of your brand strategy. Your distribution channels are also part of your brand strategy. And what you communicate visually and verbally are part of your brand strategy, too. We like it the way we have chosen our logo.
</p>
<p>
Our color palet will be a mild gradient, with distribution from the top to the bottom of the page. The color will be very mild, as we have a lot of product images! And thus, it will be annoyning to have a <i>rainbow</i> of colors in the website.
</p>
<img src="images/pallete.png" alt="Color Pallete">
</section>
</section>
<section id="UserGuide">
<h2>User Guide</h2>
<p>
The case studies presented below come in the help of the user and stand out as detailed website explanations.
</p>
<section id="CaseStudy">
<h3>Case Study</h3>
<p>
The below case studies will work as a user documentation or (might be) a FAQ section, as it provides a detailed description of how to fully use certain site components, functions and futures.
Considering there is no prototype for the aplication, the images presented resembles some storyboards of the actual application and might prove different from the end product. Still, they will do the job of presenting the correct way of handling main functionalities.
</p>
<section id="CaseStudyNr1">
<h4> Return a product </h4>
<p>
As any other client of a e-commerce website, the right to return a product is guarantee by law. Our website will also provide this functionality in full cooperation with the retailer.
Give the nature of our way of purchasing, the products bought through the 'Batch Purchases' will require some sort of interaction between us, the retailer and the client.
The first step for the user is to get into his product list of automatically bought goods, by accesing the buton from the menu, as shown in image 1 below.
<p>
<img src="images/case0.png" alt="Menu item click" id="PurchasesMenu">
<span class="caption">Image 1. Menu item</span>
</p>
This will redirect the user to the page that will show him the list of recently automatically bought products, where he could find out further details about all the purchases he made to this moment with this account. In the right side of each product you will find two icons, the one pointed by the red arrow in Image 2 is the "Return Button" ( to be more exact, this is the button <img src="images/return.png" alt="Return product" height="17px" width="17px">). Click it, and it will redirect you to the return form of that specific form.
<p>
<img src="images/case1a.png" alt="Purchase page">
<span class="caption">Image 2. Recent Purchases Page</span>
</p>
This button will redirect you to the "Return page", where you'll see a form in which some informations must be filled in order to make the return. The Image 3 presented bellow is an example of how the form will look alike:
<p>
<img src="images/case2.png" alt="Return page and form">
<span class="caption">Image 3. Return Page</span>
</p>
Some of the data will be automatically filled from your user account details, but not all. Some fields will be mandatory to fill. Once that is done, click the butom "Send form" as displayed in Image 3 from above. This will send the form to an operator, that will come back to you (via email or phone) in the interval of 24 to 48 hours.
</p>
<p>
You can always view the status of your return, by clicking on the drop-down menu item named "Returns as showed in the Image 4 presented below.
<p>
<img src="images/case3.png" alt="Get to returns">
<span class="caption">Image 4. Return Page menu item</span>
</p>
This will redirect you to a list of products you've returned (or they are in the process of return), select one of them and click it. You will be redirected to a page in which you could view the status of the return (as highlighted in the red rectangle) and where you could Cancel the return request (by clicking the "Delete" button as shown by the mouse pointer in Image 5).
<p>
<img src="images/case4.png" alt="Return status">
<span class="caption">Image 5. Return Page for a specific product</span>
</p>
</p>
</section>
<section id="CaseStudyNr2">
<h4> Ask for warranty </h4>
<p>
Any user is entitled to ask for a warranty to his product (as long that the product still owns one). For this the user has a special button and a form form which he could ask such a service. How the pick-up of the product will function, is up to the retailer, which is the one responsible for the warranty. What is our interest in this business, is to provide the user with the necessary tools to contact the seller.
First of all, in order to ask for the warranty, the user must go to his list of purchases that he can find by clicking the "Purchases" menu item, as in <a href="#PurchasesMenu">Image 1</a>.
This will direct him to the "Product List" in which for each bought product some information is displayed and there is a service button (<img src="images/service.png" alt="Service product" height="17px" width="17px">) that will redirect the user to the warranty page. Further directions can be seen in Image 6, the red arrow points to the "Service Button"
<p>
<img src="images/case5.png" alt="Return status">
<span class="caption">Image 6. Service Button</span>
</p>
The button will redirect the user to a Warranty Request Form which will have some data fill in automatically. The user must "Send" the form, and that would be all. See Image 7 below:
<p>
<img src="images/case6.png" alt="Return status">
<span class="caption">Image 7. Service Form</span>
</p>
</section>
<section id="CaseStudyNr3">
<h4> Place a instant purchase </h4>
<p>
One of our services is to find the lowest prices on the market and show it to the user. Our profit is made by redirecting the usert to the retailer through out a profit share link (which will grant us some profits).
If a user wants to acces this feature he will have to select the menu item "Best Prices" from the main menu, which will redirect the user to the page responsible with searching for best deals available. This menu item is dsplayed in Image 8 below:
<p>
<img src="images/case7.png" alt="Return status">
<span class="caption">Image 8. Best Prices menu item</span>
</p>
From here, the user can search the wanted products, by placing some tags in the search box and hit the search button. The results will be displayed below. The arrow marked 1 from Image 9 will show the search box as I've described. The search can be more targeted by using the filter option from the left of the page (also marked by arrow number 2). This will allow the user to filter multiple results and have a more complex targeting.
Once the user had seen the wanted product he can hit the button named "See offers" as shown in Image 9 by the mouse pointer.
<p>
<img src="images/case8.png" alt="Return status">
<span class="caption">Image 9. Best Prices search page</span>
</p>
Once the user had clicked "See offers" he will be redirecte to the product page (view Image 10 for more), here he can se more product images (shown by arrow marked 1) gathered from different retailers. He is also provided with a short product description, shown by the arrow marked 2. A graph is shown in the center of the page containing historical prices.
The price, and the retailer is also displayed (see arrow marked 3).
If the user is willing to see more, he can click the green button in the left of the product which will redirect him to the retailers page where he can make the purchase (the mouse pointer from Image 10 shows this button).
</p>
<p>
<img src="images/case9.png" alt="Return status">
<span class="caption">Image 10. Product Page</span>
</p>
</section>
</section>
<section id="VideoDemonstration">
<h3>Video Demonstration</h3>
<p>The video demonstration we created highlights the main site features and tends to bring a markatable aspect onto our company.</p>
...video
</section>
</section>
<section id="bibliography">
<h2>Bibliography</h2>
"Structured Systems Analysis: Tools and Techniques", by C. Gane and T. Sarson (New York: IST, Inc., 1977).
<dl>
<dt id="ref-Books">Books</dt>
<dd property="schema:citation" typeof="schema:ScholarlyArticle" resource="http://www.scribd.com/doc/224608514/The-Full-New-York-Times-Innovation-Report">
<cite property="schema:name"><a href="http://www.scribd.com/doc/224608514/The-Full-New-York-Times-Innovation-Report">The
Full New York Times Innovation Report</a></cite>,
by
<span property="schema:author" typeof="schema:Person">
<span property="schema:name">New York Times</span>
</span>;
<time property="schema:datePublished" datetime="2014-03" datatype="xsd:gYearMonth">2014 Mar</time>.
</dd>
<dt id="ref-Apache">Apache</dt>
<dd property="schema:citation" typeof="schema:WebPage" resource="https://wiki.apache.org/nutch/FrontPage">
<cite property="schema:name"><a href="https://wiki.apache.org/nutch/FrontPage">Apache Nutch Documentation</a></cite>.
</dd>
<dd property="schema:citation" typeof="schema:WebPage" resource="https://spark.apache.org/documentation.html">
<cite property="schema:name"><a href="https://spark.apache.org/documentation.html">Apache Spark Documentation</a></cite>.
</dd>
<dd property="schema:citation" typeof="schema:WebPage" resource="https://apacheignite.readme.io/v2.0/docs/memory-and-cache-metrics">
<cite property="schema:name"><a href="https://apacheignite.readme.io/v2.0/docs/memory-and-cache-metrics">Apache Ignite Documentation</a></cite>.
</dd>
<dd property="schema:citation" typeof="schema:WebPage" resource="http://lucene.apache.org/solr/features.html">
<cite property="schema:name"><a href="http://lucene.apache.org/solr/features.html">Solr Documentation</a></cite>.
</dd>
<dt id="ref-Untangled">Untangled</dt>
<dd property="schema:citation" typeof="schema:WebPage" resource="https://www.untangle.com/category/case-studies/">
<cite property="schema:name"><a href="https://www.untangle.com/category/case-studies/">Untangled Case Studies </a></cite>.
</dd>
<dt id="ref-AWS">AWS</dt>
<dd property="schema:citation" typeof="schema:WebPage" resource="https://aws.amazon.com/elasticloadbalancing/features/?nc=sn&loc=2">
<cite property="schema:name"><a href="https://aws.amazon.com/elasticloadbalancing/features/?nc=sn&loc=2">Elastic Load Balancer</a></cite>.
</dd>
<dd property="schema:citation" typeof="schema:WebPage" resource="https://aws.amazon.com/oracle/">
<cite property="schema:name"><a href="https://aws.amazon.com/oracle/">AWS and Oracle</a></cite>.
</dd>
<dd property="schema:citation" typeof="schema:WebPage" resource="https://aws.amazon.com/caching/cdn/">
<cite property="schema:name"><a href="https://aws.amazon.com/caching/cdn/">CDN from AWS</a></cite>.
</dd>
<dd property="schema:citation" typeof="schema:WebPage" resource="https://pay.amazon.com/us/developer/documentation">
<cite property="schema:name"><a href="https://pay.amazon.com/us/developer/documentation">Amazon pay documentation</a></cite>.
</dd>
<dt id="ref-RDFa">Oracle</dt>
<dd property="schema:citation" typeof="schema:WebPage" resource="https://docs.oracle.com/en/database/">
<cite property="schema:name"><a href="https://docs.oracle.com/en/database/">Oracle DB for transactions</a></cite>.
</dd>
<dt id="ref-schema.org">schema.org</dt>
<dd property="schema:citation" typeof="schema:WebPage" resource="http://schema.org/">
<cite property="schema:name"><a href="http://schema.org/">schema.org</a></cite>.
</dd>
</dl>
</section>
<section id="Annexes">
<h2>Annexes</h2>
<p>Anexes, none yet!</p>
</section>
<hr>
<p style="text-align: center;">
SDBIS2 - 2019
</p>
</body>
</html>