Category: Firemonkey

TMS WEB Core v1.2 tips & tricks part 5: Accessing microphone and camera

Over the years, increasingly new web APIs have given web developers access to areas of the operating system that were originally only accessible from native applications. This is also the case with the microphone and camera that is attached to a desktop computer or integrated in laptops, tablets and smartphones.

As the main vision and mission of TMS WEB Core is to bring OO RAD component based development for web client applications, it is expected that TMS WEB Core also brings easy to use components for using your device microphone or cameras (yes that is plural as it is possible to select between front and back cameras on a mobile device for example) from a TMS WEB Core application.

Security / privacy

An important first remark to make is that before a web client application can access a microphone or camera, user authorization is required. So, clearly, no web client application can secretly in the back listen to your conversations or watch what you are doing! Together with this authorization is also the requirement that the web application comes from a HTTPS URL so there is a reliable identification of the author or company hosting the application possible. So, when your TMS WEB Core application runs from a HTTPS domain and you want to access a microphone or camera, the user will see a prompt to allow this access:

Microphone

To use the microphone, we have introduced the non-visual component TWebMediaCapture. This component can be used to capture audio via the microphone when TWebMediaCapture.Capture is set to mctAudio. The standard recording mode is mrmManual. This means that recording is started by calling WebMediaCapture.Start and the recording is stopped by calling WebMediaCapture.Stop. After the recording is stopped, the data that was recorded is asynchronously returned via the event OnStopCapture that has the event signature:

TMediaCaptureCloseEvent = procedure(Sender: TObject; ABinary: TJSUint8Array; ABase: string) of object;

This event returns the audio via a JavaScript array or as a base64 encoded string.

An important remark is that different browsers might have a different default audio file format. In the Chrome browser this is the webm format for example, in Firefox it is ogg.

With this component, we have created a TMS WEB Core dictaphone app. You can discover this application here: https://download.tmssoftware.com/tmsweb/demos/tmsweb_Dictaphone/

The code to start and stop the recording is simply done from a click on an image:


procedure TForm1.WebImageControl1Click(Sender: TObject);
begin
  if not FRecording then
  begin
    WebMediaCapture1.Start;
    FRecording := True;
    WebImageControl1.URL := GetStopImage;
  end
  else
  begin
    WebMediaCapture1.Stop;
    FRecording := False;
    WebImageControl1.URL := GetRecordImage;
  end;
end;

The code to capture the recorded data is here:

procedure TForm1.WebMediaCapture1StopCapture(Sender: TObject;
  ABinary: TJSUint8Array; ABase: string);
begin
  if WebIndexedDbClientDataset1.RecordCount > 0 then
    WebIndexedDbClientDataset1.RecNo := 1;

  WebIndexedDbClientDataset1.Insert;
  WebIndexedDbClientDataset1.FieldByName('descr').AsString := WebEdit1.Text;
  WebIndexedDbClientDataset1.FieldByName('time').AsDateTime := Now;
  WebIndexedDbClientDataset1.FieldByName('base').AsString := ABase;
  WebIndexedDbClientDataset1.Post;
  WebEdit1.Text := 'SoundClip ' + IntToStr(WebIndexedDbClientDataset1.RecordCount + 1);
end;

This code shows another interesting concept, that is to store the recorded audio in an IndexedDB dataset. That is a dataset hosted in your browser which is private and accessible at a later time, so you can listen again to captured audio snippets when you open the web application again at a later time.

One more interesting option is to let the microphone record automatically when a certain audio level is reached. To use the TWebMediaCapture this way, set WebMediaCapture.RecordingMode to mrmAudio. With the properties WebMediaCaptature.Sensitivity, WebMediaRecorder.FFTSize, WebMediaRecorder.SmoothingTimeConstant it can be controlled at what noise level and what duration of the noise level the recording will start and stop.

Camera

Thanks to the new TWebCamera component, taking pictures from a TMS WEB Core web client application is equally easy. Drop the component on the form and start the camera by calling WebCamera.Start. To take a picture, call one of the three properties depending on the format you wish for the captured image data:

property WebCamera.SnapShotAsBase64: string;
property WebCamera.SnapShotAsImageData: TJSImageData;
property WebCamera.SnapShotAsUint8Array: TJSUint8Array;

A very simple use is to call WebCamera.SnapShotAsBase64 and assign the result to a HTML image element, i.e. what is wrapped in TWebImageControl via:

WebImageControl1.URL := WebCamera1.SnapShotAsBase64;

The image format returned van be selected between JPEG or PNG with WebCamera.BaseFormat.

As mentioned, TWebCamera can handle multiple cameras. To select a camera, the property WebCamera.CameraType can be used that can select between rear, front or default camera. To detect the cameras available, the TWebCamera.CameraDevices collection can be used. You can use:

for i := 0 to WebCamera.CameraDevices.Count - 1 do
begin
  WebListBox1.Items.Add(WebCamera.CameraDevices[i].Name);
end;

For camera & microphone use, we have published 3 demos you can discover at: https://www.tmssoftware.com/site/tmswebcoreintro.asp#demos
There is even a camera demo that integrates with a QR and barcode scanning library so you can detect and decode pictures taken from a QR code or barcode. Of course, the TMS WEB Core distribution contains the full source code of these demos

Discover these and more exciting features in TMS WEB Core that let you create web client applications with unprecedented capabilities.

You can download the trial version, go ahead with the standalone version you purchased or with TMS WEB Core and additional tools that are all included in TMS ALL-ACCESS. Our team looks forward to all your comments, feedback, feature-requests to help steering the development of TMS WEB Core to the next stages!

Read More

Read More

How to use Image1.Bitmap.BitmapChanged;

Bitmap.BitmapChanged; is protected in FMX.Graphics so I cannot use the procedure. Useing a TImage or TImageControler I am drawing a line but the line does not show. I am using this snippet: imgc1.Bitmap.Canvas.BeginScene; imgc1.Bitmap.Canvas.DrawLine(FStartPoint,FEndPoint, 100); imgc1.Bitmap.Canvas.EndScene; imgc1.Bitmap.BitmapChanged; // the original example said that this would redraw the image. In my CE Rio IDE the BitmapChanged is undefind. How can I use it? Draw the line. IDE cannot find BitmapChanged.
Read More

Android – The case of the vanishing file

I’m relatively new to the Android development platform, coming from the Windows desktop. One of the great things about recent editions of Delphi is it’s ability target numerous other platforms. One of the worst things about recent versions of Delphi is it’s ability to target platforms you are not as familiar with. If your use case is atypical, you can quickly find yourself swimming in shark infested waters. You never know what is going to byte you, a bug in the platform, a bug in Delphi RTL or FMX code, a bug in your code, or just your lack of knowledge about the platform. Today I was trying to figure out why files I was saving to the Documents folder on my Android LG5 were visible from the Android File Manager, but not present when I connected the phone to a Windows, or Ubuntu desktop. Turns out there is a long outstanding bug in Android that google has not even acknowledged. In order to see new files when connecting to a device using the MTP protocol, you may need to re-boot the Android device before the files appear. Indeed once I did so, my files appeared and I could copy them off to my PC. You may want to comment on this motion to get the issue re-opened and addressed. Hopefully this helps someone else avoid spending a lot of time scratching their head…
Read More

UWP and Windows App Platform

UWP and Windows App Platform As you have probably seen, over the last month there has been some discussion around UWP (Universal Windows Platform) and the current focus, or change of focus, for Microsoft in terms of how developers should build applications for their mainstream operating system, Windows 10. There have been claims and counterclaims, making the entire debate quite interesting. Needless to say, Microsoft directions for building Windows applications is of critical importance to most of our customers, and also for us as at Embarcadero, given RAD Studio has a strong focus on Windows development. For a summary of Microsoft point of view, you can read this interview by Mary Jo Foley of Microsoft Corporate Vice President of the Windows Developer Platform Kevin Gallo: https://www.zdnet.com/article/microsoft-wants-to-close-the-uwp-win32-divide-with-windows-apps/ But let me start from the beginning and recap Microsoft plans (and how Embarcadero addressed those). UWP Holy Grail and WinRT A few years back, when Microsoft introduced Windows 10 for desktop the company still had high hopes for its twin Windows 10 Phone operating system. We know the phone ended up being a big failure for Microsoft, but at the time they were betting heavily on it. The idea behind UWP was to create a new common API and core operating system infrastructure that would allow the same application to run on all versions of Windows (including also Xbox and HoloLens). Technically, Windows 10 came with a new "core" called WinRT (initially introduced in Windows 8), a subsystem you can target with a specific API. This is not a .NET API, but more of a native, C++ type of API. Side Note: Turns out that WinRT is "invocation compatible" with COM and it can be easily mapped to Delphi interfaces -- which is what Embarcadero did offering direct support for this new platform API. There were other key tenets: increased security (as apps run more in isolation and are safer in their memory management), more stability (with more responsive UIs and async processes), and that could allow Windows become a more stable ecosystem. So why didn't all developers jumped to UWP and WinRT? Microsoft offered a variety of programming languages (C# and .NET, C++, even JavaScript) and they expected large adoption. However, UWP came with its own UI design guidelines (wide spaced controls, full screen mode) and technical specifications (almost all code requiring to be asynchronous) that hampered development. Nice, modern, but quite different from any other API, platform and UX -- and so difficult to adopt by developer, particularly if the target is more than one platform. Basically none of the existing code was ready to run there, regardless of the programming language you picked. And doing a full rewrite is not something developers (and their companies) are keen on doing. Cross the Bridge or Stay on the Trusted Side? After the initial dismal adoption (which allowed developers to publish app on the store -- but a store that wasn't well maintained) Microsoft started promoting a set of "bridges" to allow bringing existing code to the new UWP platform. There was a bridge for Android apps support on Windows Phone, one for Objective-C conversion, and one to allow Win32 native apps to become part of UWP. Now these were non-universal UWP platform apps, given they could run only on one of the versions of Windows 10, desktop or mobile. This ended up only adding to the confusion. Moreover, Microsoft kept indicating these bridges were meant to help converting your applications to the new model, one form and one module at a time -- rather than all at once. But the destination was still supposed to be a complete rewrite of your code. "If you have a million lines of code, you want to move to the new platform over time", Microsoft was officially stating. But it was obvious developers didn't really want to do ANY rewrite of their code for moving it from Windows to Windows 10! Considering all old applications kept running smoothly and the brand new features offered by the operating system were fairly limited. Now again, RAD Studio (as well as Visual Studio) started offering support for the Desktop Bridge, but our goal was only letting developers target new APIs available only on the WinRT side (like notifications or BLE) and be able to deploy applications on the Windows Store -- for easier distribution, taking advantage of low commissions, and additional monetization options (via subscriptions). Is Microsoft Giving Up? This brings us to today and the recent "announcements" by Microsoft. An interesting POV is at: https://mspoweruser.com/uwp-is-dead-because-windows-apps-are-dead/ The company has apparently just come to the realization that no matter their insistence developers are not going to rewrite applications to target the Windows operating system. In fact they keep using WinForms and MFC (on Microsoft developer tools side) or VCL (on the RAD studio one) or other native libraries. These days Microsoft is clearly downplaying UWP as a development model. Notice that there are also reports of the fact XBox apps are now being built with Electron, a JavaScript desktop library: https://www.onmsft.com/news/new-electron-powered-xbox-app-leaks-hours-before-e3 The claim development for the Windows platform has stopped seems fairly exaggerated. There are still a lot of business applications written for Windows, and also real time control systems and games. While a lot of consumer apps moved to the web and mobile, investment on Windows development is still a significant portion of the IT budget -- even though maintenance of existing applications is likely a significant part of it. In any case, this is only one side of the scenario. The APPX model of the desktop bridge implies apps run "partially sand-boxed". They cannot be elevated with admin permission, they can access only their "view" of the registry and operating system. Granted, they can still do damage, but the store vetting procedure should also help filtering out the bad guys. The concept around UWP / APPX is applications can be more easily isolated (and with .NET Core even have each their own copy of any required library). They can be installed with no admin permission and removed leaving a clean (or almost completely clean) operating system, mimicking the experience users have on phones. Now this is not just surviving to the UWP model, but Microsoft has double its effort with the MSIX technology. This sounds and it is being promoted as an "installation" technology, but it is in fact the next interaction of APPX. MSIX apps can be distributed via Windows Store or within an enterprise distribution model, allow for extra security and what Microsoft originally called "virtualization" and now dubs as "containerization" -- to piggyback on a common trend. The idea apps should live more and more in isolation is also backed by changes at the .NET level, with the idea (a key element of .NET Core) that each app should ship with its own version of .NET rather than relying on the version of library the operating system provides -- with the big advantage of reducing dependencies and allowing users to update .NET without fearing to break existing applications on their systems. And What About RAD Studio? This is how I understand the future of the Windows App platform -- a future that allows Delphi and C++Builder VCL applications to play on par with Microsoft own UI libraries and ecosystem, with a much higher degree of compatibility with existing code and investment. Windows 10 as target platform is alive and growing (within the desktop OS total, if not in absolute terms) and the VCL with its support for Windows API, COM, WinRT and APPX model has everything you need to target the platform. Our libraries integrate Windows 10 feature, High DPI support, and compatibility in a way that's unparalleled in the industry -- without forgetting a large ecosystem of great third party controls!
Read More

convert vcl project to fmx Delphi 10.3

I try to use Listview in my fmx project , and grab text from it Like vcl , How can I using it in fmx project not in vcl ? procedure TForm1.Button1Click(Sender: TObject); var item: TListItem; begin item := ListView1.Items.Add; item.Caption := 'Mohamed'; item.SubItems.Add('Akram'); item.SubItems.Add('055002011'); item.ImageIndex := 0; end; procedure TForm1.CaptionclickClick(Sender: TObject); begin Memo1.Lines.Add('caption : ' + ListView1.Items[ListView1.Selected. Index].Caption); end; procedure TForm1.FirstitemclickClick(Sender: TObject); begin Memo1.Lines.Add('First sub item : ' + ListView1.Items[ListView1.Selected. Index].SubItems[0]); end; procedure TForm1.SeconditemclickClick(Sender: TObject); begin Memo1.Lines.Add('Second sub item : ' + ListView1.Items[ListView1.Selected. Index].SubItems[1]); end; https://6.top4top.net/p_1260py56w1.jpg
Read More

How to bring a TreeViewItem in the visible (scroll) area

I'm looking for a way to navigate by code to an item in a tree view. The object should be moved into the visible area. I could not find a method in either TTreeView or TTreeViewItem. The following approach does not work under all circumstances because the item position is not always updated: procedure TfmxMain.MakeItemVisible(Item: TTreeViewItem); begin trvMyTreeView.ViewportPosition := TPointF.Create(min(Item.Position.X - trvSlideGroups.ClientWidth / 2, 0), min(Item.Position.Y - trvSlideGroups.ClientHeight / 2, 0)); end;
Read More

How to refresh TListView LiveBinding with TAdapterBindSource

I am using Delphi 10.3.1 (Firemonkey FMX) to build android and iOS app. I have a TListView, live binding with a AdapterBindSource. My problem is: new records does not appear after Adapter refreshed. ============== I created a TObjectList, added 3 objects to it I created a TBindSourceAdapter by passing a TObjectList to create it. I assign the TBindSourceAdapter to AdapterBindSource1.Adapter. Then I Free the TObjectList and re-create it, add 4 newly created objects (3 of them are old records, with some data modified, 1 is a new record) I do TBindSourceAdapter.Refresh and TAdapterBindSource.Refresh Those 3 old records are refreshed successfully with modified data displayed, but the new record is not showing up in Android and iOS The same logic working fine in Windows platform ============== My logic create TObjectList first I get records from Rest Server and converted into a TObjectList TData : class(TObject) ... // a class stored some data TDataList = class(TObjectList<TData>) // then I get data from Rest Server and created FList, it is a Form private variable FList := TDataList.Create; // a private Form variable // create Tdata objects and add to FList ..... create TBindSourceAdapter, assign to AdapterBindSource var ABindSourceAdapter: TBindSourceAdapter; // .... ABindSourceAdapter := TListBindSourceAdapter<TData>.Create(self, FList, True); AdapterBindSource1.Adapter := ABindSourceAdapter; AdapterBindSource1.Active := true; then the records show on ListView which live bindings with the AdapterBindSource Refresh FList records When click on Refresh button, I trigger to get data from Rest server again, I do free the FList and re-create it FreeAndNil(FList); FList := TDataList.Create; // re-create the list, then create Tdata object and add to it again. refresh the Adapter then I refresh the adapter AdapterBindSource1.Adapter.Refresh; AdapterBindSource1.Refresh; here the 3 old records are refreshed successfully, modified data are displayed correctly, however, new record is not showing, the TListView still showing 3 records only. Notes: I did not re-create TListBindSourceAdapter and assign to AdapterBindSource1.Adapter again during refresh, the records still refreshed successfully. However, even I re-create TListBindSourceAdapter and assign to AdapterBindSource1.Adapter again, new record still does not show up, only caused memory leaking. How can I resolve this? is there something I missing like to refresh the TListView? Or my BindSourceAdapter refresh logic is wrong? Thanks for any help.
Read More

Even perpetual Delphi licenses require an active maintenance support to allow re-install on a fresh machine

Last month, people found out that retroactively, Embarcadero has changed the terms of the license agreements on products sold with a perpetual license: In order to re-install those products, often a bump in license count is needed. That bump now requires an active maintenance subscription which has a substantial yearly cost. This is yet another […] … Read More

Read More